iFormer: Inception Transformer (NeurIPS 2022 Oral)
This is a PyTorch implementation of iFormer proposed by our paper "Inception Transformer".
Image Classification
1. Requirements
torch>=1.7.0; torchvision>=0.8.1; timm==0.5.4; fvcore; apex-amp (if you want to use fp16);
data prepare: ImageNet with the following folder structure, you can extract ImageNet by this script.
βimagenet/
βββtrain/
β βββ n01440764
β β βββ n01440764_10026.JPEG
β β βββ n01440764_10027.JPEG
β β βββ ......
β βββ ......
βββval/
β βββ n01440764
β β βββ ILSVRC2012_val_00000293.JPEG
β β βββ ILSVRC2012_val_00002138.JPEG
β β βββ ......
β βββ ......
Main results on ImageNet-1K
| Model | #params | FLOPs | Image resolution | acc@1 | Model |
|---|---|---|---|---|---|
| iFormer-S | 20M | 4.8G | 224 | 83.4 | model/config/log |
| iFormer-B | 48M | 9.4G | 224 | 84.6 | model/config/log |
| iFormer-L | 87M | 14.0G | 224 | 84.8 | model/config/log |
Fine-tuning Results with larger resolution (384x384) on ImageNet-1K
| Model | #params | FLOPs | Image resolution | acc@1 | Model |
|---|---|---|---|---|---|
| iFormer-S | 20M | 16.1G | 384 | 84.6 | model/config/log |
| iFormer-B | 48M | 30.5G | 384 | 85.7 | model/config/log |
| iFormer-L | 87M | 45.3G | 384 | 85.8 | model/config/log |
Training
Train iformer_small on 224
python -m torch.distributed.launch --nproc_per_node=8 train.py /dataset/imagenet \
--model iformer_small -b 128 --epochs 300 --img-size 224 --drop-path 0.2 --lr 1e-3 \
--weight-decay 0.05 --aa rand-m9-mstd0.5-inc1 --warmup-lr 1e-6 --warmup-epochs 5 \
--output checkpoint --min-lr 1e-6 --experiment iformer_smallFinetune on 384 based on the pretrained checkpoint on 224
python -m torch.distributed.launch --nproc_per_node=8 fine-tune.py /dataset/imagenet \
--model iformer_small_384 -b 64 --lr 1e-5 --min-lr 1e-6 --warmup-lr 2e-8 --warmup-epochs 0 \
--epochs 20 --img-size 384 --drop-path 0.3 --weight-decay 1e-8 --mixup 0.1 --cutmix 0.1 \
--cooldown-epochs 10 --aa rand-m9-mstd0.5-inc1 --clip-grad 1.0 --output checkpoint_fine \
--initial-checkpoint checkpoint/iformer_small/model_best.pth.tar \
--experiment iformer_small_384Validation
python validate.py /dataset/imagenet --model iformer_small --checkpoint checkpoint/iformer_small/model_best.pth.tarObject Detection and Instance Segmentation
All models are based on Mask R-CNN and trained by 1xβ training schedule.
| Backbone | #Param. | FLOPs | box mAP | mask mAP |
|---|---|---|---|---|
| iFormer-S | 40M | 263G | 46.2 | 41.9 |
| iFormer-B | 67M | 351G | 48.3 | 43.3 |
Semantic Segmentation
| Backbone | Method | #Param. | FLOPs | mIoU |
|---|---|---|---|---|
| iFormer-S | FPN | 24M | 181G | 48.6 |
| iFormer-S | Upernet | 49M | 938G | 48.4 |
Bibtex
@inproceedings{
si2022inception,
title={Inception Transformer},
author={Chenyang Si and Weihao Yu and Pan Zhou and Yichen Zhou and Xinchao Wang and Shuicheng YAN},
booktitle={Advances in Neural Information Processing Systems},
year={2022}
}
Acknowledgment
Our implementation is mainly based on the following codebases. We gratefully thank the authors for their wonderful works.
pytorch-image-models, mmdetection, mmsegmentation.
Besides, Weihao Yu would like to thank TPU Research Cloud (TRC) program for the support of partial computational resources.