ergo (from the Latin sentence "Cogito ergo sum") is a command line tool that makes machine learning with Keras easier.
It can be used to:
- scaffold new projects in seconds and customize only a minimum amount of code.
- encode samples, import and optimize CSV datasets and train the model with them.
- visualize the model structure, loss and accuracy functions during training.
- determine how each of the input features affects the accuracy by differential inference.
- export a simple REST API to use your models from a server.
Installing
sudo pip3 install ergo-ai
Installing from Sources
git clone https://github.com/evilsocket/ergo.git
cd ergo
sudo pip3 install -r requirements.txt
python3 setup.py build
sudo python3 setup.py install
Enable GPU support (optional)
Make sure you have CUDA 11 and cuDNN 8.0 installed and then:
sudo pip3 uninstall tensorflow
sudo pip3 install tensorflow-gpu
Example Projects
- A planes detector from satellite imagery.
- An anti malware API for Windows.
Usage
To print the general help menu:
ergo help
To print action specific help:
ergo <action> -h
Start by printing the available actions by running ergo help, you can also print the software version (ergo, keras and tensorflow versions) and some hardware info with ergo info to verify your installation.
Creating a Project
Once ready, create a new project named example (ergo create -h to see how to customize the initial model):
ergo create example
Inside the newly created example folder, there will be three files:
prepare.py, used to preprocess your dataset and inputs (if, for instance, you're using pictures instead of a csv file).model.py, that you can change to customize the model.train.py, for the training algorithm.
By default, ergo will simply read the dataset as a CSV file, build a small neural network with 10 inputs, two hidden layers of 30 neurons each and 2 outputs and use a pretty standard training algorithm.
Exploration (optional)
Explore properties of the dataset. Ergo can generate graphs and tables that can be useful for the feature engineering of the problem.
Explore can show:
- Metrics of each feature (min, max, standard deviation) - Which can be used to discard constant features in the dataset.
- Feature correlation of each feature with the target - Which can give an idea of how good is feature is as a linear predictor.
- Feature correlation matrix.
- PCA decomposition:
- 2D projection of the data based on classes.
- Explained variance of each principal component with 90, 95 and 99 % explanation values.
- Kmeans clustering or DBSCAN clustering of the data.
- Elbow method to determine the optimal number of clusters for kmeans.
Example with a dataset some/path/data.csv:
ergo explore example --dataset some/path/data.csv -p
This will show the PCA decomposition of the dataset, saving (and optionally showing) the explained variance vs the number of principal component vectors used and the 2D projection of the dataset (colored by labels).
A full exploratory analysis can be performed using the --all flag:
ergo explore example --dataset some/path/data.csv --all
Encoding (optional)
In case you implemented the prepare_input function in the prepare.py script, ergo can be used to encode raw samples, being them executables, images, strings or whatever, into vectors of scalars that are then saved into a dataset.csv file suitable for training
Example with a folder /path/to/data which contains a pos and neg subfolders, in auto labeling mode each group of sample is labeled with its parent directory name:
ergo encode example /path/to/data
Example with a single folder and manual labeling:
ergo encode example /path/to/data --label 'some-label'
Example with a single text file containing multiple inputs, one per line:
ergo encode example /path/to/data --label 'some-label' -m
Training
After defining the model structure and the training process, you can import a CSV dataset (first column must be the label) and start training using 2 GPUs:
ergo train example --dataset /some/path/data.csv --gpus 2
This will split the dataset into a train, validation and test sets (partitioned with the --test and --validation arguments), start the training and once finished show the model statistics.
If you want to update a model and/or train it on already imported data, you can simply:
ergo train example --gpus 2
Testing
Now it's time to visualize the model structure and how the the accuracy and loss metrics changed during training (requires sudo apt-get install graphviz python3-tk):
ergo view example
If the data-test.csv file is still present in the project folder (ergo clean has not been called yet), ergo view will also show the ROC curve.
You can use the relevance command to evaluate the model on a given set (or a subset of it, see --ratio 0.1) by nulling one attribute at a time and measuring how that influenced the accuracy (feature.names is an optional file with the names of the attributes, one per line):
ergo relevance example --dataset /some/path/data.csv --attributes /some/path/feature.names --ratio 0.1
Once you're done, you can remove the train, test and validation temporary datasets with:
ergo clean example
Inference
To load the model and start a REST API for evaluation (can be customized with --address, --port, --classes and --debug options):
ergo serve example
To run an inference on a vector of scalars:
curl "http://localhost:8080/?x=0.345,1.0,0.9,..."
If you customized the prepare_input function in prepare.py (see the Encoding section), you can run an inference on a raw sample:
curl "http://localhost:8080/?x=/path/to/sample"
The input x can also be passed as a POST request:
curl --data 'x=...' "http://localhost:8080/"
Or as a file upload:
curl -F 'x=@/path/to/file' "http://localhost:8080/"
The API can also be used to perform encoding only:
curl -F 'x=@/path/to/file' "http://localhost:8080/encode"
This will return the raw features vector that can be used for inference later.
Other commands
To reset the state of a project (WARNING: this will remove the datasets, the model files and all training statistics):
ergo clean example --all
Evaluate and compare the performances of two trained models on a given dataset and (optionally) output the differences to a json file:
ergo cmp example_a example_b --dataset /path/to/data.csv --to-json diffs.json
Freeze the graph and convert the model to the TensorFlow protobuf format:
ergo to-tf example
Convert the Keras model to frugally-deep format:
ergo to-fdeep example
Optimize a dataset (get unique rows and reuse 15% of the total samples, customize ratio with the --reuse-ratio argument, customize output with --output):
ergo optimize-dataset /some/path/data.csv
License
ergo was made with