• Stars
    star
    812
  • Rank 54,634 (Top 2 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 7 years ago
  • Updated almost 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Play deep learning with CIFAR datasets

Convolutional Neural Networks for CIFAR-10

This repository is about some implementations of CNN Architecture for cifar10.

cifar10

I just use Keras and Tensorflow to implementate all of these CNN models.
(maybe torch/pytorch version if I have time)
A pytorch version is available at CIFAR-ZOO

Requirements

  • Python (3.5)
  • keras (>= 2.1.5)
  • tensorflow-gpu (>= 1.4.1)

Architectures and papers

Documents & tutorials

There are also some documents and tutorials in doc & issues/3.
Get it if you need.
You can also see the articles if you can speak Chinese.

Accuracy of all my implementations

In particular
Change the batch size according to your GPU's memory.
Modify the learning rate schedule may imporve the results of accuracy!

network GPU params batch size epoch training time accuracy(%)
Lecun-Network GTX1080TI 62k 128 200 30 min 76.23
Network-in-Network GTX1080TI 0.97M 128 200 1 h 40 min 91.63
Vgg19-Network GTX1080TI 39M 128 200 1 h 53 min 93.53
Residual-Network20 GTX1080TI 0.27M 128 200 44 min 91.82
Residual-Network32 GTX1080TI 0.47M 128 200 1 h 7 min 92.68
Residual-Network110 GTX1080TI 1.7M 128 200 3 h 38 min 93.93
Wide-resnet 16x8 GTX1080TI 11.3M 128 200 4 h 55 min 95.13
Wide-resnet 28x10 GTX1080TI 36.5M 128 200 10 h 22 min 95.78
DenseNet-100x12 GTX1080TI 0.85M 64 250 17 h 20 min 94.91
DenseNet-100x24 GTX1080TI 3.3M 64 250 22 h 27 min 95.30
DenseNet-160x24 1080 x 2 7.5M 64 250 50 h 20 min 95.90
ResNeXt-4x64d GTX1080TI 20M 120 250 21 h 3 min 95.19
SENet(ResNeXt-4x64d) GTX1080TI 20M 120 250 21 h 57 min 95.60

About LeNet and CNN training tips/tricks

LeNet is the first CNN network proposed by LeCun.
I used different CNN training tricks to show you how to train your model efficiently.

LeNet_keras.py is the baseline of LeNet,
LeNet_dp_keras.py used the Data Prepossessing [DP],
LeNet_dp_da_keras.py used both DP and the Data Augmentation[DA],
LeNet_dp_da_wd_keras.py used DP, DA and Weight Decay [WD]

network GPU DP DA WD training time accuracy(%)
LeNet_keras GTX1080TI - - - 5 min 58.48
LeNet_dp_keras GTX1080TI - - 5 min 60.41
LeNet_dp_da_keras GTX1080TI - 26 min 75.06
LeNet_dp_da_wd_keras GTX1080TI 26 min 76.23

For more CNN training tricks, see Must Know Tips/Tricks in Deep Neural Networks (by Xiu-Shen Wei)

About Learning Rate schedule

Different learning rate schedule may get different training/testing accuracy!
See ./htd, and HTD for more details.

About Multiple GPUs Training

Since the latest version of Keras is already supported keras.utils.multi_gpu_model, so you can simply use the following code to train your model with multiple GPUs:

from keras.utils import multi_gpu_model
from keras.applications.resnet50 import ResNet50

model = ResNet50()

# Replicates `model` on 8 GPUs.
parallel_model = multi_gpu_model(model, gpus=8)
parallel_model.compile(loss='categorical_crossentropy',optimizer='adam')

# This `fit` call will be distributed on 8 GPUs.
# Since the batch size is 256, each GPU will process 32 samples.
parallel_model.fit(x, y, epochs=20, batch_size=256)

About ResNeXt & DenseNet

Since I don't have enough machines to train the larger networks, I only trained the smallest network described in the paper. You can see the results in liuzhuang13/DenseNet and prlz77/ResNeXt.pytorch

   

Please feel free to contact me if you have any questions!

Citation

@misc{bigballon2017cifar10cnn,
  author = {Wei Li},
  title = {cifar-10-cnn: Play deep learning with CIFAR datasets},
  howpublished = {\url{https://github.com/BIGBALLON/cifar-10-cnn}},
  year = {2017}
}

More Repositories

1

CIFAR-ZOO

PyTorch implementation of CNNs for CIFAR benchmark
Python
695
star
2

PyTorch-CPP

PyTorch C++ inference with LibTorch
C++
327
star
3

distribuuuu

The pure and clear PyTorch Distributed Training Framework.
Python
267
star
4

Ghost

An AI search algorithm demo
C++
58
star
5

CPP-Call-Tensorflow

Calling (TensorFlow) Python Program from C++
C++
51
star
6

Deep-learning-and-practices

Version control for my deep learning course.
Python
47
star
7

Caffe2-Tutorial

Caffe2 C++/Python tutorial with full demo
Python
39
star
8

Paper_List

Paper reading list during my graduate studies
18
star
9

Crawler_Demo

This is my Crawler exercises.
CSS
16
star
10

CVPR2022-Paper-Statistics

Paper Statistics for CVPR‘22
Python
14
star
11

qduoj_install_sh

青岛大学OnlineJudge部署脚本
Shell
7
star
12

ResNet_CIFAR

Residual Network Experiments with CIFAR Datasets.
Python
6
star
13

HTD

Source code for HTD (WACV 2019)
Lua
6
star
14

codePrint

简单的代码打印系统,用于acm-icpc比赛的代码打印。
HTML
5
star
15

NCTU_NP

Version control for my network programming course.
C++
4
star
16

kosmos-2-gd

Shell
3
star
17

NCTU_AI

Artificial Intelligence Term Project Spring 2017
C++
3
star
18

Toward-AGZ

Materials for AlphaGo
3
star
19

bigballon.github.io

This is my personal blog.
SCSS
3
star
20

NCTU_CV

code for my computer vision course.
C++
3
star
21

ubuntu_rc

my ubuntu setting files
Shell
3
star
22

pretrained_models

Pretrained Residual network models
2
star
23

Model_Experiment

useless experiment of HTD
Lua
2
star
24

NCTU_TCG

Version control for my theory of computer game course.
C++
2
star
25

STQPF

Short-Term Quantitative Precipitation Forecasting@CIKM AnalytiCup 2017
Python
1
star
26

Meow

Notes for BG
1
star
27

Hiphop-Gym

Python
1
star
28

bigballon

1
star
29

Dog_challenge

Jupyter Notebook
1
star