• Stars
    star
    140
  • Rank 261,473 (Top 6 %)
  • Language
    Python
  • Created almost 5 years ago
  • Updated almost 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Pytorch implementation of our paper accepted by IJCAI 2020 -- Channel Pruning via Automatic Structure Search

Channel Pruning via Automatic Structure Search (Link).

PyTorch implementation of ABCPruner (IJCAI 2020).

Tips

Any problem, free to contact the authors via emails: [email protected] or [email protected]. Do not post issues with github as much as possible, just in case that I could not receive the emails from github thus ignore the posted issues.

Citation

If you find ABCPruner useful in your research, please consider citing:

@inproceedings{lin2020channel,
  title={Channel Pruning via Automatic Structure Search},
  author={Lin, Mingbao and Ji, Rongrong and Zhang, Yuxin and Zhang, Baochang and Wu, Yongjian and Tian, Yonghong},
  booktitle={Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)},
  pages={673 -- 679},
  year={2020}
}

Experimental Results

We provide our pruned models in the paper and their training loggers and configuration files below.

(The percentages in parentheses indicate the pruned rate)

CIFAR-10

Full Model Params Flops Channels Accuracy Pruned Model
VGG16 1.67M(88.68%) 82.81M(73.68%) 1639(61.20%) 93.08% ABCPruner-80%
ResNet56 0.39M(54.20%) 58.54M(54.13%) 1482(27.07%) 93.23% ABCPruner-70%
ResNet110 0.56M(67.41%) 89.87M(65.04%) 2701(33.28%) 93.58% ABCPruner-60%
GoogLeNet 2.46M(60.14%) 513.19M(66.56) 6150(22.19%) 94.84% ABCPruner-30%

ImageNet

Full Model Params Flops Channels Acc Top1 Acc Top5 Pruned Model
ResNet18 6.6M(43.55%) 1005.71M(44.88%) 3894(18.88%) 67.28% 87.28% ABCPruner-70%
ResNet18 9.5M(18.72%) 968.13M(46.94%) 4220(12%) 67.80% 88.00% ABCPruner-100%
ResNet34 10.52M(51.76%) 1509.76M(58.97%) 5376(25.09%) 70.45% 89.688% ABCPruner-50%
ResNet34 10.12M(53.58%) 2170.77M(41%) 6655(21.82%) 70.98% 90.053% ABCPruner-90%
ResNet50 7.35M(71.24%) 944.85M(68.68%) 20576(25.53%) 70.289% 89.631% ABCPruner-30%
ResNet50 9.1M(64.38%) 1295.4M(68.68%) 21426(19.33%) 72.582% 90.19% ABCPruner-50%
ResNet50 11.24M(56.01%) 1794.45M(56.61%) 22348(15.86%) 73.516% 91.512% ABCPruner-70%
ResNet50 11.75(54.02%) 1890.6M(54.29%) 22518(15.22%) 73.864% 91.687% ABCPruner-80%
ResNet50 18.02(29.5%) 2555.55M(38.21%) 24040(9.5%) 74.843% 92.272% ABCPruner-100%
ResNet101 12.94M(70.94%) 1975.61M(74.89%) 41316(21.56%) 74.683% 92.08% ABCPruner-50%
ResNet101 17.72M(60.21%) 3164.91M(59.78%) 43168(17.19%) 75.823% 92.736% ABCPruner-80%
ResNet152 15.62M(74.06%) 2719.47M(76.57%) 58750(22.4%) 76.004% 92.901% ABCPruner-50%
ResNet152 24.07M(60.01%) 4309.52M(62.87%) 62368(17.62%) 77.115% 93.481% ABCPruner-70%

Running Code

Requirements

  • Pytorch >= 1.0.1
  • CUDA = 10.0.0

Pre-train Models

Additionally, we provide several pre-trained models used in our experiments.

CIFAR-10

| VGG16 | ResNet56 | ResNet110 |GoogLeNet |

ImageNet

|ResNet18 | ResNet34 | ResNet50 |ResNet101 | ResNet152|

Training

python bee_imagenet.py
--data_path ../data/ImageNet2012 
--honey_model ./pretrain/resnet18.pth 
--job_dir ./experiment/resnet_imagenet 
--arch resnet
--cfg resnet18
--lr 0.01 
--lr_decay_step 75 112 
--num_epochs 150 
--gpus 0 
--calfitness_epoch 2 
--max_cycle 50 
--max_preserve 9 
--food_number 10 
--food_limit 5 
--random_rule random_pretrain

Get FLOPS & Params

python get_flops_params.py 
--data_set cifar10 
--arch resnet_cifar 
--cfg resnet56
--honey 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 # honey is the optimal pruned structure and can be found in the training logger.

Check Our Results

python bee_imagenet.py
--data_path ../data/ImageNet2012 
--job_dir ./experiment/resnet_imagenet 
--arch resnet
--cfg resnet18
--test_only
--gpus 0
--honey_model ./pretrain/resnet18.pth  #path of the pre-trained model.
--best_honey  5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 # honey is the optimal pruned structure and can be found in the training logger.     
--best_honey_s ./pruned/resnet18_pruned.pth   # path of the pruned model. 

Other Arguments

optional arguments:
  -h, --help            show this help message and exit
  --gpus GPUS [GPUS ...]
                        Select gpu_id to use. default:[0]
  --data_set DATA_SET   Select dataset to train. default:cifar10
  --data_path DATA_PATH
                        The dictionary where the input is stored.
                        default:/home/lishaojie/data/cifar10/
  --job_dir JOB_DIR     The directory where the summaries will be stored.
                        default:./experiments
  --reset               Reset the directory?
  --resume RESUME       Load the model from the specified checkpoint.
  --refine REFINE       Path to the model to be fine tuned.
  --arch ARCH           Architecture of model. default:vgg,resnet,googlenet,densenet
  --cfg CFG             Detail architecuture of model. default:vgg16, resnet18/34/50(imagenet),resnet56/110(cifar),googlenet,densenet
  --num_epochs NUM_EPOCHS
                        The num of epochs to train. default:150
  --train_batch_size TRAIN_BATCH_SIZE
                        Batch size for training. default:128
  --eval_batch_size EVAL_BATCH_SIZE
                        Batch size for validation. default:100
  --momentum MOMENTUM   Momentum for MomentumOptimizer. default:0.9
  --lr LR               Learning rate for train. default:1e-2
  --lr_decay_step LR_DECAY_STEP [LR_DECAY_STEP ...]
                        the iterval of learn rate decay. default:30
  --weight_decay WEIGHT_DECAY
                        The weight decay of loss. default:5e-4
  --random_rule RANDOM_RULE
                        Weight initialization criterion after random clipping.
                        default:default
                        optional:default,random_pretrain,l1_pretrain
  --test_only           Test only?
  --honey_model         Path to the model wait for Beepruning. default:None
  --calfitness_epoch    Calculate fitness of honey source: training epochs. default:2
  --max_cycle           Search for best pruning plan times. default:10
  --food_number         number of food to search. default:10
  --food_limit          Beyond this limit, the bee has not been renewed to become a scout bee default:5
  --honeychange_num     Number of codes that the nectar source changes each time default:2
  --best_honey          If this hyper-parameter exists, skip bee-pruning and fine-tune from this prune method default:None
  --best_honey_s        Path to the best_honey default:None
  --best_honey_past     If you want to load a resume without honey code, input your honey hode into this hyper-parameter default:None
  --honey               get flops and params of a model with specified honey(prune plan )
  --from_scratch        if this parameter exist, train from scratch 
  --warm_up             if this parameter exist, use warm up lr like DALI
  --bee_from_scratch    if this parameter exist, beepruning from scratch
  --label_smooth        if this parameter exist, use Lable smooth criterion
  --split_optimizer     if this parameter exist, split the weight parameter that need weight decay