Awesome CIFAR Zoo

Status: Archive ( Final test with PyTorch 1.7 and no longer maintained, I would recommend you to use pycls powered by FAIR, which is a simple and flexible codebase for image classification )

This repository contains the pytorch code for multiple CNN architectures and improve methods based on the following papers, hope the implementation and results will helpful for your research!!

• Architecure
  • (lenet) LeNet-5, convolutional neural networks
  • (alexnet) ImageNet Classification with Deep Convolutional Neural Networks
  • (vgg) Very Deep Convolutional Networks for Large-Scale Image Recognition
  • (resnet) Deep Residual Learning for Image Recognition
  • (preresnet) Identity Mappings in Deep Residual Networks
  • (resnext) Aggregated Residual Transformations for Deep Neural Networks
  • (densenet) Densely Connected Convolutional Networks
  • (senet) Squeeze-and-Excitation Networks
  • (bam) BAM: Bottleneck Attention Module
  • (cbam) CBAM: Convolutional Block Attention Module
  • (genet) Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
  • (sknet) SKNet: Selective Kernel Networks
• Regularization
  • (shake-shake) Shake-Shake regularization
  • (cutout) Improved Regularization of Convolutional Neural Networks with Cutout
  • (mixup) mixup: Beyond Empirical Risk Minimization
• Learning Rate Scheduler
  • (cos_lr) SGDR: Stochastic Gradient Descent with Warm Restarts
  • (htd_lr) Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification

Requirements and Usage

Requirements

• Python (>=3.6)
• PyTorch (>=1.1.0)
• Tensorboard(>=1.4.0) (for visualization)
• Other dependencies (pyyaml, easydict)

pip install -r requirements.txt

Usage

simply run the cmd for the training:

## 1 GPU for lenet
CUDA_VISIBLE_DEVICES=0 python -u train.py --work-path ./experiments/cifar10/lenet

## resume from ckpt
CUDA_VISIBLE_DEVICES=0 python -u train.py --work-path ./experiments/cifar10/lenet --resume

## 2 GPUs for resnet1202
CUDA_VISIBLE_DEVICES=0,1 python -u train.py --work-path ./experiments/cifar10/preresnet1202

## 4 GPUs for densenet190bc
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u train.py --work-path ./experiments/cifar10/densenet190bc

## 1 GPU for vgg19 inference
CUDA_VISIBLE_DEVICES=0 python -u eval.py --work-path ./experiments/cifar10/vgg19

We use yaml file config.yaml to save the parameters, check any files in ./experimets for more details.
You can see the training curve via tensorboard, tensorboard --logdir path-to-event --port your-port.
The training log will be dumped via logging, check log.txt in your work path.

Results on CIFAR

Vanilla architectures

With additional regularization

PS: the default data augmentation methods are RandomCrop + RandomHorizontalFlip + Normalize,
and the √ means which additional method be used. 🍰

PS: shake_resnet26_2x64d achieved 97.71% test accuracy with cutout and mixup!!
It's cool, right?

With different LR scheduler

Acknowledgments

Provided codes were adapted from

• kuangliu/pytorch-cifar
• bearpaw/pytorch-classification
• timgaripov/swa
• xgastaldi/shake-shake
• uoguelph-mlrg/Cutout
• facebookresearch/mixup-cifar10
• BIGBALLON/cifar-10-cnn
• BayesWatch/pytorch-GENet
• Jongchan/attention-module
• pppLang/SKNet

Feel free to contact me if you have any suggestions or questions, issues are welcome,
create a PR if you find any bugs or you want to contribute. 😊

Citation

@misc{bigballon2019cifarzoo,
  author = {Wei Li},
  title = {CIFAR-ZOO: PyTorch implementation of CNNs for CIFAR dataset},
  howpublished = {\url{https://github.com/BIGBALLON/CIFAR-ZOO}},
  year = {2019}
}