niku-server/README.md

niku-server

Don't forget to install chopy!!!! As long as it imports from the same shell you run make launch, you're good.

Makefile commands

• sudo make install: Install prerequisites, configure nginx/postgres
• sudo make unistall: Remove nginx/postgres configuration
• sudo make clean: Reset the environment and database to postinstall
• make launch: Launch an instance with production parameters
• make debug: Launch an instance with debug parameters, listening on localhost:8080
• make stop: Halt the components started by launch or debug

System architecture

nignx

Very simple nginx configuration, in conf/nginx.conf. Basically just serves static pcaps and forwards everything to uwsgi.

uwsgi

make launch will start an instance of uwsgi, serving app.py. It will use the configuration parameters from config.py, and log to data/log/uwsgi.log.

This is basically just an API wrapper around the chopy database.

chopy

make launch will run launch-chopy.py to load the configuration from config.py and launch a chopy instance, by default logging to data/log/chopy.log. All the folders expected by chopy will be put in a data folder. By default you should dump pcaps into data/pcap_dump, and they will be sorted into the database and data/pcap_split.

API

Most of the interfaces take their parameters as a json encoded object, passed in the query string, for example GET /api/search?{}. I'm very sorry for this.

• GET /api/search - search the database and index. Provide a dictionary of parameters that are the keyword arguments to chopy.search.search. Returns the matching IDs, one per line.
• GET /api/metadata - retrieve stream metadata. Provide a list of ids for which to retrieve the metadata. Returns the metadata as a series of json-encoded dictionaries, one per line. No guarantee is made about the order of the returned values, check the id of each. The metadata is in the same form as the chopy database, but as dictionaries insead of relations.
• GET /pcap/<path> - replace <path> with the filename attribute from a stream's metadata to download its individual pcap.

Tags, Services, and Hosts

Tags, services, and hosts use a similar API to get/set/delete data.

• GET /api/<kind>/get - Retrive all the known resources of the given kind Returns each resource as a separate json-encoded dictionary, one per line.

• GET /api/<kind>/set - Create or update the given resource. Provide as a dictionary all the identifier and data arguments for the resource.

• GET /api/<kind>/del - Delete a given resource. Provide as a dictionary all the identifier arguments for the resource.

• For kind tag, use identifier arguments connection and text. There are no data arguments.

• For kind service use identifier arguments protocol, host, and port, and name as a data argument.

• For kind host, use boot_time as an identifier argument and name as a data argument.