• Stars
    star
    168
  • Rank 225,507 (Top 5 %)
  • Language
    Python
  • License
    MIT License
  • Created over 4 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Official Pytorch implementation of ReBias (Learning De-biased Representations with Biased Representations), ICML 2020

Learning De-biased Representations with Biased Representations (ICML 2020)

Official Pytorch implementation of ReBias | Paper

Hyojin Bahng1, Sanghyuk Chun2, Sangdoo Yun2, Jaegul Choo3, Seong Joon Oh2

1 Korea university
2 Clova AI Research, NAVER Corp.
3 KAIST

Many machine learning algorithms are trained and evaluated by splitting data from a single source into training and test sets. While such focus on in-distribution learning scenarios has led to interesting advancement, it has not been able to tell if models are relying on dataset biases as shortcuts for successful prediction (e.g., using snow cues for recognising snowmobiles), resulting in biased models that fail to generalise when the bias shifts to a different class. The cross-bias generalisation problem has been addressed by de-biasing training data through augmentation or re-sampling, which are often prohibitive due to the data collection cost (e.g., collecting images of a snowmobile on a desert) and the difficulty of quantifying or expressing biases in the first place. In this work, we propose a novel framework to train a de-biased representation by encouraging it to be different from a set of representations that are biased by design. This tactic is feasible in many scenarios where it is much easier to define a set of biased representations than to define and quantify bias. We demonstrate the efficacy of our method across a variety of synthetic and real-world biases; our experiments show that the method discourages models from taking bias shortcuts, resulting in improved generalisation.

VIDEO

Updates

  • 26 Jun, 2020: Initial upload.

Summary of code contributions

The code repository contains the implementations of our method (ReBias) as well as prior de-biasing methods empirically compared in the paper. Specifically, we provide codes for:

We support training and evaluation of above methods on the three diverse datasets and tasks. See trainer.py and evaluator.py for the unified framework. Supported datasets and tasks are:

In this implementation, we set Adam as the default optimiser for the reproducibility. However, we strongly recommend using a better optimiser AdamP [3] by --optim AdamP for future researches. We refer the official repository of AdamP for interested users.

Installation

MNIST and ImageNet experiments

Our implementations are tested on the following libraries with Python3.7.

fire
munch
torch==1.1.0
torchvision==0.2.2.post3
adamp

Install dependencies using the following command.

pip install -r requirements.txt

Action recognition experiments

For action recoginition tasks, we implement the baselines upon the official implementation of SlowFast.

NOTE: We will not handle the issues from action recognition experiments.

Please follow the official SlowFast instruction: https://github.com/facebookresearch/SlowFast/blob/master/INSTALL.md

Dataset preparation

Biased MNIST

Biased MNIST is a colour-biased version of the original MNIST. datasets/colour_mnist.py downloads the original MNIST and applies colour biases on images by itself. No extra preparation is needed on the user side.

ImageNet

We do not provide a detailed instruction for collecting the ImageNet (ILSVRC2015) dataset. Please follow the usual practice.

ImageNet-A and ImageNet-C

To further measure the generalisability of de-biasing methods, we perform evaluations on the ImageNet-A ([4] Hendrycks, et al. 2019) and ImageNet-C ([5] Hendrycks, et al. 2019) as well. The datasets are available at https://github.com/hendrycks/natural-adv-examples and https://github.com/hendrycks/robustness, respectively.

NOTE: We implement the ImageNet-C evaluator separately to this implementation, and do not provide the implementation here. Please refer to [5] for details.

Kinetics and Mimetics

We use two datasets for action recognition: Kinetics and Mimetics ([6] Weinzaepfel, et al. 2019).

Kinetics and Mimetics datasets are available at:

NOTE: We will not handle the issues from action recognition experiments.

How to run

Biased MNIST results in Table 1

Table1

Main experiments for the Biased MNIST are configured in main_biased_mnist.py. Note that we have reported the average of three runs in the main paper; the standard deviations are reported in the appendix.

NOTE: We do not provide HEX [7] implementation which is significantly different from the other baselines. It does not require any biased model but containes pre-defined handcrafted feature extractor named NGLCM. Thus, instead of providing HEX under the unified framework, we have implemented it separately. Please refer to official HEX implementation for details.

ReBias (ours)

For the better results with AdamP

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.999 --optim AdamP

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.997 --optim AdamP

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.995 --optim AdamP

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.99 --optim AdamP

For the original numbers,

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.999

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.997

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.995

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.99

Vanilla & Biased

By setting f_lambda_outer and g_lambda_inner to 0, f and g are trained separately without minimax optimization.

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.999
    --f_lambda_outer 0
    --g_lambda_inner 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.997
    --f_lambda_outer 0
    --g_lambda_inner 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.995
    --f_lambda_outer 0
    --g_lambda_inner 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.99
    --f_lambda_outer 0
    --g_lambda_inner 0

Learned Mixin

In our experiments, we first pretrain g networks for the Learned Mixin and optimize F using the fixed g. Hence, n_g_pretrain_epochs and n_g_update are set to 5 and 0, respectively.

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.999
    --outer_criterion LearnedMixin
    --g_lambda_inner 0
    --n_g_pretrain_epochs 5
    --n_g_update 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.997
    --outer_criterion LearnedMixin
    --g_lambda_inner 0
    --n_g_pretrain_epochs 5
    --n_g_update 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.995
    --outer_criterion LearnedMixin
    --g_lambda_inner 0
    --n_g_pretrain_epochs 5
    --n_g_update 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.99
    --outer_criterion LearnedMixin
    --g_lambda_inner 0
    --n_g_pretrain_epochs 5
    --n_g_update 0

RUBi

RUBi updates F and g simultaneously but separately. We set g_lambda_inner to 0 for only updating g network using the classification loss.

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.999
    --outer_criterion RUBi
    --g_lambda_inner 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.997
    --outer_criterion RUBi
    --g_lambda_inner 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.995
    --outer_criterion RUBi
    --g_lambda_inner 0

python main_biased_mnist.py --root /path/to/your/dataset
    --train_correlation 0.99
    --outer_criterion RUBi
    --g_lambda_inner 0

ImageNet results in Table 2

Model Biased (Standard acc) Unbiased (Texture clustering) ImageNet-A [4]
Vanilla (ResNet18) 90.8 88.8 24.9
Biased (BagNet18) 67.7 65.9 19.5
Stylised ImageNet [8] 88.4 86.6 24.6
LearnedMixin [1] 67.9 65.9 18.8
RUBi [2] 90.5 88.6 27.7
ReBias (ours) 91.9 90.5 29.6

Main experiments for ImageNet are configured in main_imagenet.py . Note that we have reported the average of three runs in the main paper; the standard deviations are reported in the appendix.

ReBias (ours)

Our default settings to kernel radius for HSIC is median, which need to compute pair-wise distances in training dataset. We set the kernel radius by median of the distances. Thus, rbf_sigma_x and rbf_sigma_y are set to median, and we recompute the kernel radius for every epoch.

To aviod unnecessary computations, we set rbf_sigma_x and rbf_sigma_y to 1 for other methods.

python main_imagenet.py --train_root /path/to/your/imagenet/train
    --val_root /path/to/your/imagenet/val
    --imageneta_root /path/to/your/imagenet_a
    --optim AdamP

For the original numbers,

python main_imagenet.py --train_root /path/to/your/imagenet/train
    --val_root /path/to/your/imagenet/val
    --imageneta_root /path/to/your/imagenet_a

Vanilla & Biased

python main_imagenet.py --train_root /path/to/your/imagenet/train
    --val_root /path/to/your/imagenet/val
    --imageneta_root /path/to/your/imagenet_a
    --f_lambda_outer 0
    --g_lambda_inner 0
    --rbf_sigma_x 1
    --rbf_sigma_y 1

Learned Mixin

python main_imagenet.py --train_root /path/to/your/imagenet/train
    --val_root /path/to/your/imagenet/val
    --imageneta_root /path/to/your/imagenet_a
    --outer_criterion LearnedMixin
    --n_g_pretrain_epochs 30
    --n_g_update 0
    --rbf_sigma_x 1
    --rbf_sigma_y 1

RUBi

python main_imagenet.py --train_root /path/to/your/imagenet/train
    --val_root /path/to/your/imagenet/val
    --imageneta_root /path/to/your/imagenet_a
    --outer_criterion RUBi
    --g_lambda_inner 0
    --rbf_sigma_x 1
    --rbf_sigma_y 1

Stylised ImageNet

We train Stylised ImageNet by training the vanilla model with the original ImageNet train dataset and the stylised train dataset (IN + SIN in [8]). Please follow the instruction in the following repositories to build the stylised ImageNet training dataset.

Run the following command

python main_imagenet.py --train_root /path/to/your/imagenet_and_stylised_imagenet/train
    --val_root /path/to/your/imagenet/val
    --imageneta_root /path/to/your/imagenet_a
    --f_lambda_outer 0
    --g_lambda_inner 0

Note that the command is identical to the Vanilla & Biased case, except that the training set is now replaced by the Stylized ImageNet.

Action recognition results in Table 3

NOTE: We will not handle the issues from action recognition experiments.

Model Biased (Kinetics) Unbiased (Mimetics) [6]
Vanilla (3D-ResNet) 54.5 18.9
Biased (2D-ResNet) 50.7 18.4
LearnedMixin [1] 11.4 12.3
RUBi [2] 23.6 13.4
ReBias (ours) 55.8 22.4

ReBias (ours)

python main_action.py --train_root /path/to/your/kinetics/train
    --train_annotation_file /path/to/your/kinetics/train_annotion
    --eval_root /path/to/your/mimetics/train
    --eval_annotation_file /path/to/your/kinetics/train_annotion

Learned Mixin

python main_action.py --train_root /path/to/your/kinetics/train
    --train_annotation_file /path/to/your/kinetics/train_annotion
    --eval_root /path/to/your/mimetics/train
    --eval_annotation_file /path/to/your/kinetics/train_annotion
    --outer_criterion LearnedMixin
    --n_g_pretrain_epochs 30
    --n_g_update 0
    --rbf_sigma_x 1
    --rbf_sigma_y 1 

RUBi

python main_action.py --train_root /path/to/your/kinetics/train
    --train_annotation_file /path/to/your/kinetics/train_annotion
    --eval_root /path/to/your/mimetics/train
    --eval_annotation_file /path/to/your/kinetics/train_annotion
    --outer_criterion RUBi
    --g_lambda_inner 0
    --rbf_sigma_x 1
    --rbf_sigma_y 1

License

This project is distributed under MIT license.

Copyright (c) 2020-present NAVER Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

How to cite

@inproceedings{bahng2019rebias,
    title={Learning De-biased Representations with Biased Representations},
    author={Bahng, Hyojin and Chun, Sanghyuk and Yun, Sangdoo and Choo, Jaegul and Oh, Seong Joon},
    year={2020},
    booktitle={International Conference on Machine Learning (ICML)},
}

References

  • [1] Clark, C., Yatskar, M., and Zettlemoyer, L. Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases. EMNLP, 2019.
  • [2] Cadene, R., Dancette, C., Cord, M., Parikh, D., et al. Rubi: Reducing unimodal biases for visual question answering. In Advances in Neural Information Processing Systems, 2019.
  • [3] Heo, B., Chun, S., Oh, S. J., Han, D., Yun, S., Uh, Y., and Ha, J. W. Slowing Down the Weight Norm Increase in Momentum-based Optimizers. arXiv preprint arXiv:2006.08217, 2020.
  • [4] Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, D. Natural adversarial examples. arXiv preprint arXiv:1907.07174, 2019.
  • [5] Hendrycks, D., Dietterich, T. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. ICLR, 2019.
  • [6] Weinzaepfel, P. and Rogez, G. Mimetics: Towards understanding human actions out of context. arXiv preprint arXiv:1912.07249, 2019.
  • [7] Wang, H., He, Z., and Xing, E. P. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019.
  • [8] Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., and Brendel, W. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.

More Repositories

1

donut

Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022
Python
5,573
star
2

deep-text-recognition-benchmark

Text recognition (optical character recognition) with deep learning methods, ICCV 2019
Jupyter Notebook
3,692
star
3

stargan-v2

StarGAN v2 - Official PyTorch Implementation (CVPR 2020)
Python
3,478
star
4

CRAFT-pytorch

Official implementation of Character Region Awareness for Text Detection (CRAFT)
Python
3,024
star
5

CutMix-PyTorch

Official Pytorch implementation of CutMix regularizer
Python
1,211
star
6

voxceleb_trainer

In defence of metric learning for speaker recognition
Python
1,029
star
7

WCT2

Software that can perform photorealistic style transfer without the need of any post-processing steps.
Python
869
star
8

synthtiger

Official Implementation of SynthTIGER (Synthetic Text Image Generator), ICDAR 2021
Python
482
star
9

tunit

Rethinking the Truly Unsupervised Image-to-Image Translation - Official PyTorch Implementation (ICCV 2021)
Python
452
star
10

rexnet

Official Pytorch implementation of ReXNet (Rank eXpansion Network) with pretrained models
Python
451
star
11

AdamP

AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights (ICLR 2021)
Python
411
star
12

overhaul-distillation

Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)
Python
409
star
13

cord

CORD: A Consolidated Receipt Dataset for Post-OCR Parsing
384
star
14

cutblur

Rethinking Data Augmentation for Image Super-resolution (CVPR 2020)
Jupyter Notebook
379
star
15

wsolevaluation

Evaluating Weakly Supervised Object Localization Methods Right (CVPR 2020)
Python
331
star
16

assembled-cnn

Tensorflow implementation of "Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network"
Python
329
star
17

generative-evaluation-prdc

Code base for the precision, recall, density, and coverage metrics for generative models. ICML 2020.
Python
239
star
18

ext_portrait_segmentation

Python
238
star
19

ClovaCall

ClovaCall dataset and Pytorch LAS baseline code (Interspeech 2020)
Python
218
star
20

fewshot-font-generation

The unified repository for few-shot font generation methods. This repository includes FUNIT (ICCV'19), DM-Font (ECCV'20), LF-Font (AAAI'21) and MX-Font (ICCV'21).
Python
203
star
21

stargan-v2-tensorflow

StarGAN v2 - Official Tensorflow Implementation (CVPR 2020)
Python
187
star
22

EXTD_Pytorch

Official EXTD Pytorch code
Python
187
star
23

CLEval

CLEval: Character-Level Evaluation for Text Detection and Recognition Tasks
Python
185
star
24

TedEval

TedEval: A Fair Evaluation Metric for Scene Text Detectors
Python
176
star
25

aasist

Official PyTorch implementation of "AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks"
Python
167
star
26

SATRN

Official Tensorflow Implementation of SATRN (CVPR Workshop WTDDLE 2020)
Python
162
star
27

lffont

Official PyTorch implementation of LF-Font (Few-shot Font Generation with Localized Style Representations and Factorization) AAAI 2021
Python
156
star
28

bros

Python
156
star
29

som-dst

SOM-DST: Efficient Dialogue State Tracking by Selectively Overwriting Memory (ACL 2020)
Python
150
star
30

mxfont

Official PyTorch implementation of MX-Font (Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts) ICCV 2021
Python
148
star
31

dmfont

Official PyTorch implementation of DM-Font (ECCV 2020)
Python
133
star
32

rainbow-memory

Official pytorch implementation of Rainbow Memory (CVPR 2021)
Python
119
star
33

FocusSeq2Seq

[EMNLP 2019] Mixture Content Selection for Diverse Sequence Generation (Question Generation / Abstractive Summarization)
Python
113
star
34

attention-feature-distillation

Official implementation for (Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching, AAAI-2021)
Python
111
star
35

frostnet

FrostNet: Towards Quantization-Aware Network Architecture Search
Python
106
star
36

webvicob

Official Implementation of Web-based Visual Corpus Builder (Webvicob), ICDAR 2023
Python
101
star
37

length-adaptive-transformer

Official Pytorch Implementation of Length-Adaptive Transformer (ACL 2021)
Python
99
star
38

spade

Python
81
star
39

embedding-expansion

Official MXNet implementation of "Embedding Expansion: Augmentation in Embedding Space for Deep Metric Learning" (CVPR 2020)
Python
76
star
40

symmetrical-synthesis

Official Tensorflow implementation of "Symmetrical Synthesis for Deep Metric Learning" (AAAI 2020)
Python
71
star
41

units

Python
70
star
42

lookwhostalking

Look Who’s Talking: Active Speaker Detection in the Wild
Python
70
star
43

subword-qac

Subword Language Model for Query Auto-Completion
Python
67
star
44

ssmix

Official PyTorch Implementation of SSMix (Findings of ACL 2021)
Python
60
star
45

SSUL

[NeurIPS 2021] SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning
Python
59
star
46

BESTIE

[CVPR 2022] Beyond Semantic to Instance Segmentation: Weakly-Supervised Instance Segmentation via Semantic Knowledge Transfer and Self-Refinement
Python
55
star
47

PointWSSIS

[CVPR2023] The Devil is in the Points: Weakly Semi-Supervised Instance Segmentation via Point-Guided Mask Representation
Python
55
star
48

c3_sinet

Python
52
star
49

puridiver

Official PyTorch Implementation of PuriDivER CVPR 2022.
Python
45
star
50

EResFD

Lightweight Face Detector from CLOVA
Python
44
star
51

minimal-rnr-qa

[NAACL 2021] Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering
Python
36
star
52

ECLIPSE

(CVPR 2024) ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning
Python
34
star
53

group-transformer

Official code for Group-Transformer (Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model, COLING-2020).
Python
25
star
54

ProxyDet

Official implementation of the paper "ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detection"
Python
22
star
55

GeNAS

Official pytorch implementation for GeNAS: Neural Architecture Search with Better Generalization
Python
15
star
56

meev

Python
12
star
57

pkm-transformers

Official implementation of PKM-augmented language models (Findings of EMNLP 2020)
9
star
58

DCutMix

DCutMix official repo
Python
8
star
59

TVQ-VAE

Official pytorch implementation for TVQ-VAE
Jupyter Notebook
8
star
60

textual-kd-slu

Official Implementation of Textual KD SLU (ICASSP 2021)
Python
6
star
61

vat-d

Official Implementation of VAT-D
Python
5
star
62

ActiveASR_AugCR

Repositoty for Efficient Active Learning for Automatic Speech Recognition via Augmented Consistency Regularization
3
star
63

WSSS-BED

Rethinking Saliency-Guided Weakly-Supervised Semantic Segmentation
Python
1
star