EvadeML-Zoo
The goal of this project:
- Several datasets ready to use: MNIST, CIFAR-10, ImageNet-ILSVRC and more.
- Pre-trained state-of-the-art models to attack. [See details].
- Existing attacking methods: FGSM, BIM, JSMA, Deepfool, Universal Perturbations, Carlini/Wagner-L2/Li/L0 and more. [See details].
- Visualization of adversarial examples.
- Existing defense methods as baseline.
The code was developed on Python 2, but should be runnable on Python 3 with tiny modifications.
Please follow the instructions to reproduce the Feature Squeezing results.
1. Install dependencies.
pip install -r requirements_cpu.txtIf you are going to run the code on GPU, install this list instead:
pip install -r requirements_gpu.txt2. Fetch submodules.
git submodule update --init --recursive3. Download pre-trained models.
mkdir downloads; curl -sL https://github.com/mzweilin/EvadeML-Zoo/releases/download/v0.1/downloads.tar.gz | tar xzv -C downloads4. (Optional) Download the SVHN dataset and pre-trained model.
python datasets/svhn_dataset/download_svhn_data.py
curl -sL https://github.com/mzweilin/EvadeML-Zoo/releases/download/v0.1/svhn_model_weights.tar.gz | tar xzv5. Usage of python main.py
usage: python main.py [-h] [--dataset_name DATASET_NAME] [--model_name MODEL_NAME]
[--select [SELECT]] [--noselect] [--nb_examples NB_EXAMPLES]
[--balance_sampling [BALANCE_SAMPLING]] [--nobalance_sampling]
[--test_mode [TEST_MODE]] [--notest_mode] [--attacks ATTACKS]
[--clip CLIP] [--visualize [VISUALIZE]] [--novisualize]
[--robustness ROBUSTNESS] [--detection DETECTION]
[--detection_train_test_mode [DETECTION_TRAIN_TEST_MODE]]
[--nodetection_train_test_mode] [--result_folder RESULT_FOLDER]
[--verbose [VERBOSE]] [--noverbose]
optional arguments:
-h, --help show this help message and exit
--dataset_name DATASET_NAME
Supported: MNIST, CIFAR-10, ImageNet, SVHN.
--model_name MODEL_NAME
Supported: cleverhans, cleverhans_adv_trained and
carlini for MNIST; carlini and DenseNet for CIFAR-10;
ResNet50, VGG19, Inceptionv3 and MobileNet for
ImageNet; tohinz for SVHN.
--select [SELECT] Select correctly classified examples for the
experiement.
--noselect
--nb_examples NB_EXAMPLES
The number of examples selected for attacks.
--balance_sampling [BALANCE_SAMPLING]
Select the same number of examples for each class.
--nobalance_sampling
--test_mode [TEST_MODE]
Only select one sample for each class.
--notest_mode
--attacks ATTACKS Attack name and parameters in URL style, separated by
semicolon.
--clip CLIP L-infinity clip on the adversarial perturbations.
--visualize [VISUALIZE]
Output the image examples for each attack, enabled by
default.
--novisualize
--robustness ROBUSTNESS
Supported: FeatureSqueezing.
--detection DETECTION
Supported: feature_squeezing.
--detection_train_test_mode [DETECTION_TRAIN_TEST_MODE]
Split into train/test datasets.
--nodetection_train_test_mode
--result_folder RESULT_FOLDER
The output folder for results.
--verbose [VERBOSE] Stdout level. The hidden content will be saved to log
files anyway.
--noverbose
5. Example.
python main.py --dataset_name MNIST --model_name carlini \
--nb_examples 2000 --balance_sampling \
--attacks "FGSM?eps=0.1;" \
--robustness "none;FeatureSqueezing?squeezer=bit_depth_1;" \
--detection "FeatureSqueezing?squeezers=bit_depth_1,median_filter_2_2&distance_measure=l1&fpr=0.05;"Cite this work
You are encouraged to cite the following paper if you use EvadeML-Zoo for academic research.
@inproceedings{xu2018feature,
title={{Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks}},
author={Xu, Weilin and Evans, David and Qi, Yanjun},
booktitle={Proceedings of the 2018 Network and Distributed Systems Security Symposium (NDSS)},
year={2018}
}