BalancedMetaSoftmax - Classification

Code for the paper "Balanced Meta-Softmax for Long-Tailed Visual Recognition" on long-tailed visual recognition datasets.

Balanced Meta-Softmax for Long-Tailed Visual Recognition
Jiawei Ren, Cunjun Yu, Shunan Sheng, Xiao Ma, Haiyu Zhao, Shuai Yi, Hongsheng Li
NeurIPS 2020

Snapshot

def balanced_softmax_loss(labels, logits, sample_per_class, reduction):
    """Compute the Balanced Softmax Loss between `logits` and the ground truth `labels`.
    Args:
      labels: A int tensor of size [batch].
      logits: A float tensor of size [batch, no_of_classes].
      sample_per_class: A int tensor of size [no of classes].
      reduction: string. One of "none", "mean", "sum"
    Returns:
      loss: A float tensor. Balanced Softmax Loss.
    """
    spc = sample_per_class.type_as(logits)
    spc = spc.unsqueeze(0).expand(logits.shape[0], -1)
    logits = logits + spc.log()
    loss = F.cross_entropy(input=logits, target=labels, reduction=reduction)
    return loss

Requirements

Python 3
PyTorch (version == 1.4)
yaml
higher(version == 0.2.1)

Training

End-to-end Training

Base model (Representation Learning)

python main.py --cfg ./config/CIFAR10_LT/softmax_imba200.yaml

Alternatively, you may download a pretrained model here and put it in the corresponding log folder.

Balanced Softmax

python main.py --cfg ./config/CIFAR10_LT/balanced_softmax_imba200.yaml

Decoupled Training

After obataining the base model, train the classifier with the following command:

Balanced Softmax

python main.py --cfg ./config/CIFAR10_LT/decouple_balanced_softmax_imba200.yaml

BALMS

python main.py --cfg ./config/CIFAR10_LT/balms_imba200.yaml

Evaluation

Model evaluation can be done using the following command:

python main.py --cfg ./config/CIFAR10_LT/balms_imba200.yaml --test

Experiment Results

The results could be slightly different from the results reported in the paper, since we originally used an internal repository for the experiments in the paper.

Dataset	Backbone	Method	Accuracy	download
CIFAR-10 (IF 200)	ResNet-32	Softmax	74.0	model \| log
CIFAR-10 (IF 200)	ResNet-32	Balanced Softmax (end-to-end)	79.8	model \| log
CIFAR-10 (IF 200)	ResNet-32	Balanced Softmax (decouple)	81.8	model \| log
CIFAR-10 (IF 200)	ResNet-32	BALMS	82.2	model \| log
CIFAR-100 (IF 200)	ResNet-32	Softmax	41.2	model \| log
CIFAR-100 (IF 200)	ResNet-32	Balanced Softmax (end-to-end)	46.7	model \| log
CIFAR-100 (IF 200)	ResNet-32	Balanced Softmax (decouple)	47.2	model \| log
CIFAR-100 (IF 200)	ResNet-32	BALMS	48.0	model \| log
ImageNet-LT	ResNet-10	Softmax	34.8	model \| log
ImageNet-LT	ResNet-10	BALMS	41.6	model \| log
Places-LT	ResNet-152	Softmax	30.2	model \| log
Places-LT	ResNet-152	BALMS	38.3	model \| log

Cite BALMS

@inproceedings{
    Ren2020balms,
    title={Balanced Meta-Softmax for Long-Tailed Visual Recognition},
    author={Jiawei Ren and Cunjun Yu and Shunan Sheng and Xiao Ma and Haiyu Zhao and Shuai Yi and Hongsheng Li},
    booktitle={Proceedings of Neural Information Processing Systems(NeurIPS)},
    month = {Dec},
    year={2020}
}

Instance Segmentation

For BALMS on instance segmentation, please try out this repo.

Reference

The code is based on classifier-balancing.
CIFAR-LT dataset is from A Strong Single-Stage Baseline for Long-Tailed Problems
ResNet-32 is from BBN
Cutout augmentation is from Cutout
CIFAR auto augmentation is from AutoAugment

jiawei-ren/BalancedMetaSoftmax-Classification

jiawei-ren

Reviews

Repository Details