Curated imbalanced learning papers, codes, and libraries

Language: [English] [中文]

Class-imbalance (also known as the long-tail problem) is the fact that the classes are not represented equally in a classification problem, which is quite common in practice. For instance, fraud detection, prediction of rare adverse drug reactions and prediction gene families. Failure to account for the class imbalance often causes inaccurate and decreased predictive performance of many classification algorithms. Imbalanced learning aims to tackle the class imbalance problem to learn an unbiased model from imbalanced data.

Inspired by awesome-machine-learning. In this repository:

Frameworks and libraries are grouped by programming language.
Research papers are grouped by research field.
There are numerous papers in this field of research, so this list is not intended to be exhaustive.
We aim to keep only the "awesome" works that either have a good impact or have been published in reputed top conferences/journals.

Note:

⭐ Please leave a STAR if you like this project! ⭐
If you find any incorrect / inappropriate / outdated content, please kindly consider opening an issue or a PR.
We would greatly appreciate your contribution to this list, and you will appear in the contributors✨!

What's new:

Updated section Graph Learning.
Add a package imbalanced-ensemble [Github][Documentation].

Check out Zhining's other open-source projects!

_{Imbalanced-Ensemble [PythonLib]}

_{Machine Learning [Awesome]}

_{Self-paced Ensemble [ICDE]}

_{Meta-Sampler [NeurIPS]}

Table of Contents
1. Frameworks and Libraries
- 1.1 Python
- 1.2 R
- 1.3 Java
- 1.4 Scalar
- 1.5 Julia
2. Research Papers
3. Miscellaneous
- 3.1 Datasets
- 3.2 Github Repositories
Contributors ✨

1. Frameworks and Libraries

1.1 Python

imbalanced-ensemble [Github][Documentation][Gallery][Paper]

NOTE: written in python, easy to use.
- imbalanced-ensemble is a Python toolbox for quick implementing and deploying ensemble learning algorithms on class-imbalanced data. It is featured for:
  - (i) Unified, easy-to-use APIs, detailed documentation and examples.
  - (ii) Capable for multi-class imbalanced learning out-of-box.
  - (iii) Optimized performance with parallelization when possible using joblib.
  - (iv) Powerful, customizable, interactive training logging and visualizer.
  - (v) Full compatibility with other popular packages like scikit-learn and imbalanced-learn.
- Currently (v0.1.4), it includes more than 15 ensemble algorithms based on re-sampling and cost-sensitive learning (e.g., SMOTEBoost/Bagging, RUSBoost/Bagging, AdaCost, EasyEnsemble, BalanceCascade, SelfPacedEnsemble, ...).
imbalanced-learn [Github][Documentation][Paper]

NOTE: written in python, easy to use.
- imbalanced-learn is a python package offering a number of re-sampling techniques commonly used in datasets showing strong between-class imbalance. It is compatible with scikit-learn and is part of scikit-learn-contrib projects.
- Currently (v0.8.0), it includes 21 different re-sampling techniques, including over-sampling, under-sampling and hybrid ones (e.g., SMOTE, ADASYN, TomekLinks, NearMiss, OneSideSelection, SMOTETomek, ...)
- This package also provides many utilities, e.g., Batch generator for Keras/TensorFlow, see API reference.
smote_variants [Documentation][Github] - A collection of 85 minority over-sampling techniques for imbalanced learning with multi-class oversampling and model selection features (All writen in Python, also support R and Julia).

1.2 R

smote_variants [Documentation][Github] - A collection of 85 minority over-sampling techniques for imbalanced learning with multi-class oversampling and model selection features (All writen in Python, also support R and Julia).
caret [Documentation][Github] - Contains the implementation of Random under/over-sampling.
ROSE [Documentation] - Contains the implementation of ROSE (Random Over-Sampling Examples).
DMwR [Documentation] - Contains the implementation of SMOTE (Synthetic Minority Over-sampling TEchnique).

1.3 Java

KEEL [Github][Paper] - KEEL provides a simple GUI based on data flow to design experiments with different datasets and computational intelligence algorithms (paying special attention to evolutionary algorithms) in order to assess the behavior of the algorithms. This tool includes many widely used imbalanced learning techniques such as (evolutionary) over/under-resampling, cost-sensitive learning, algorithm modification, and ensemble learning methods.

NOTE: wide variety of classical classification, regression, preprocessing algorithms included.

1.4 Scalar

undersampling [Documentation][Github] - A Scala library for under-sampling and their ensemble variants in imbalanced classification.

1.5 Julia

smote_variants [Documentation][Github] - A collection of 85 minority over-sampling techniques for imbalanced learning with multi-class oversampling and model selection features (All writen in Python, also support R and Julia).

2. Research Papers

2.1 Surveys

Learning from imbalanced data (IEEE TKDE, 2009, 6000+ citations) [Paper]
- Highly cited, classic survey paper. It systematically reviewed the popular solutions, evaluation metrics, and challenging problems in future research in this area (as of 2009).
Learning from imbalanced data: open challenges and future directions (2016, 900+ citations) [Paper]
- This paper concentrates on the open issues and challenges in imbalanced learning, i.e., extreme class imbalance, imbalance in online/stream learning, multi-class imbalanced learning, and semi/un-supervised imbalanced learning.
Learning from class-imbalanced data: Review of methods and applications (2017, 900+ citations) [Paper]
- A recent exhaustive survey of imbalanced learning methods and applications, a total of 527 papers were included in this study. It provides several detailed taxonomies of existing methods and also the recent trend of this research area.

2.2 Ensemble Learning

2.2.1 General ensemble

Self-paced Ensemble (ICDE 2020, 20+ citations) [Paper][Code][Slides][Zhihu/知乎][PyPI]

NOTE: versatile solution with outstanding performance and computational efficiency.
MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler (NeurIPS 2020) [Paper][Code][Video][Zhihu/知乎]

NOTE: learning an optimal sampling policy directly from data.
Exploratory Undersampling for Class-Imbalance Learning (IEEE Trans. on SMC, 2008, 1300+ citations) [Paper]

NOTE: simple but effective solution.
- EasyEnsemble [Code]
- BalanceCascade [Code]

2.2.2 Boosting-based

AdaBoost (1995, 18700+ citations) [Paper][Code] - Adaptive Boosting with C4.5
DataBoost (2004, 570+ citations) [Paper] - Boosting with Data Generation for Imbalanced Data
SMOTEBoost (2003, 1100+ citations) [Paper][Code] - Synthetic Minority Over-sampling TEchnique Boosting
MSMOTEBoost (2011, 1300+ citations) [Paper] - Modified Synthetic Minority Over-sampling TEchnique Boosting
RAMOBoost (2010, 140+ citations) [Paper] [Code] - Ranked Minority Over-sampling in Boosting
RUSBoost (2009, 850+ citations) [Paper] [Code] - Random Under-Sampling Boosting
AdaBoostNC (2012, 350+ citations) [Paper] - Adaptive Boosting with Negative Correlation Learning
EUSBoost (2013, 210+ citations) [Paper] - Evolutionary Under-sampling in Boosting

2.2.3 Bagging-based

Bagging (1996, 20000+ citations) [Paper][Code] - Bagging predictor
Diversity Analysis on Imbalanced Data Sets by Using Ensemble Models (2009, 400+ citations) [Paper]
- UnderBagging [Code]
- OverBagging [Code]
- SMOTEBagging [Code]

2.2.4 Cost-sensitive ensemble

AdaCost (ICML 1999, 800+ citations) [Paper][Code] - Misclassification Cost-sensitive boosting
AdaUBoost (NIPS 1999, 100+ citations) [Paper][Code] - AdaBoost with Unequal loss functions
AsymBoost (NIPS 2001, 700+ citations) [Paper][Code] - Asymmetric AdaBoost and detector cascade

2.3 Data resampling

2.3.1 Over-sampling

ROS [Code] - Random Over-sampling
SMOTE (2002, 9800+ citations) [Paper][Code] - Synthetic Minority Over-sampling TEchnique
Borderline-SMOTE (2005, 1400+ citations) [Paper][Code] - Borderline-Synthetic Minority Over-sampling TEchnique
ADASYN (2008, 1100+ citations) [Paper][Code] - ADAptive SYNthetic Sampling
SPIDER (2008, 150+ citations) [Paper][Code(Java)] - Selective Preprocessing of Imbalanced Data
Safe-Level-SMOTE (2009, 370+ citations) [Paper][Code(Java)] - Safe Level Synthetic Minority Over-sampling TEchnique
SVM-SMOTE (2009, 120+ citations) [Paper][Code] - SMOTE based on Support Vectors of SVM
MDO (2015, 150+ citations) [Paper][Code] - Mahalanobis Distance-based Over-sampling for Multi-Class imbalanced problems.

NOTE: See more over-sampling methods at smote-variants.

2.3.2 Under-sampling

RUS [Code] - Random Under-sampling
CNN (1968, 2100+ citations) [Paper][Code] - Condensed Nearest Neighbor
ENN (1972, 1500+ citations) [Paper] [Code] - Edited Condensed Nearest Neighbor
TomekLink (1976, 870+ citations) [Paper][Code] - Tomek's modification of Condensed Nearest Neighbor
NCR (2001, 500+ citations) [Paper][Code] - Neighborhood Cleaning Rule
NearMiss-1 & 2 & 3 (2003, 420+ citations) [Paper][Code] - Several kNN approaches to unbalanced data distributions.
CNN with TomekLink (2004, 2000+ citations) [Paper][Code(Java)] - Condensed Nearest Neighbor + TomekLink
OSS (2007, 2100+ citations) [Paper][Code] - One Side Selection
EUS (2009, 290+ citations) [Paper] - Evolutionary Under-sampling
IHT (2014, 130+ citations) [Paper][Code] - Instance Hardness Threshold

2.3.3 Hybrid-sampling

A Study of the Behavior of Several Methods for Balancing Training Data (2004, 2000+ citations) [Paper]

NOTE: extensive experimental evaluation involving 10 different over/under-sampling methods.
- SMOTE-Tomek [Code]
- SMOTE-ENN [Code]
SMOTE-RSB (2012, 210+ citations) [Paper][Code] - Hybrid Preprocessing using SMOTE and Rough Sets Theory
SMOTE-IPF (2015, 180+ citations) [Paper][Code] - SMOTE with Iterative-Partitioning Filter

2.4 Cost-sensitive Learning

CSC4.5 (2002, 420+ citations) [Paper][Code(Java)] - An instance-weighting method to induce cost-sensitive trees
CSSVM (2008, 710+ citations) [Paper][Code(Java)] - Cost-sensitive SVMs for highly imbalanced classification
CSNN (2005, 950+ citations) [Paper][Code(Java)] - Training cost-sensitive neural networks with methods addressing the class imbalance problem.

2.5 Deep Learning

2.5.1 Surveys

A systematic study of the class imbalance problem in convolutional neural networks (2018, 330+ citations) [Paper]
Survey on deep learning with class imbalance (2019, 50+ citations) [Paper]

NOTE: a recent comprehensive survey of the class imbalance problem in deep learning.

2.5.2 Graph Data Mining

TAM: Topology-Aware Margin Loss for Class-Imbalanced Node Classification (ICML 2022) [Paper][Code]
GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networks (WSDM 2021) [Paper][Code]
Topology-Imbalance Learning for Semi-Supervised Node Classification (NeurIPS 2021) [Paper][Code]
GraphENS: Neighbor-Aware Ego Network Synthesis for Class-Imbalanced Node Classification (ICLR 2022) [Paper][Code]
LTE4G: Long-Tail Experts for Graph Neural Networks (CIKM 2022) [Paper][Code]
Multi-Class Imbalanced Graph Convolutional Network Learning (IJCAI 2020) [Paper]

2.5.3 Hard example mining

Training region-based object detectors with online hard example mining (CVPR 2016, 840+ citations) [Paper][Code] - In the later phase of NN training, only do gradient back-propagation for "hard examples" (i.e., with large loss value)

2.5.4 Loss function engineering

Focal loss for dense object detection (ICCV 2017, 2600+ citations) [Paper][Code (detectron2)][Code (unofficial)] - A uniform loss function that focuses training on a sparse set of hard examples to prevents the vast number of easy negatives from overwhelming the detector during training.

NOTE: elegant solution, high influence.
Training deep neural networks on imbalanced data sets (IJCNN 2016, 110+ citations) [Paper] - Mean (square) false error that can equally capture classification errors from both the majority class and the minority class.
Deep imbalanced attribute classification using visual attention aggregation (ECCV 2018, 30+ citation) [Paper][Code]
Imbalanced deep learning by minority class incremental rectification (TPAMI 2018, 60+ citations) [Paper] - Class Rectification Loss for minimizing the dominant effect of majority classes by discovering sparsely sampled boundaries of minority classes in an iterative batch-wise learning process.
Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss (NIPS 2019, 10+ citations) [Paper][Code] - A theoretically-principled label-distribution-aware margin (LDAM) loss motivated by minimizing a margin-based generalization bound.
Gradient harmonized single-stage detector (AAAI 2019, 40+ citations) [Paper][Code] - Compared to Focal Loss, which only down-weights "easy" negative examples, GHM also down-weights "very hard" examples as they are likely to be outliers.
Class-Balanced Loss Based on Effective Number of Samples (CVPR 2019, 70+ citations) [Paper][Code] - a simple and generic class-reweighting mechanism based on Effective Number of Samples.
Influence-Balanced Loss for Imbalanced Visual Classification (ICCV 2021) [Paper][Code]
AutoBalance: Optimized Loss Functions for Imbalanced Data (NeurIPS 2021) [Paper]
Label-Imbalanced and Group-Sensitive Classification under Overparameterization (NeurIPS 2021) [Paper][Code]

2.5.5 Meta-learning

Learning to model the tail (NIPS 2017, 70+ citations) [Paper] - Transfer meta-knowledge from the data-rich classes in the head of the distribution to the data-poor classes in the tail.
Learning to reweight examples for robust deep learning (ICML 2018, 150+ citations) [Paper][Code] - Implicitly learn a weight function to reweight the samples in gradient updates of DNN.

NOTE: representative work to solve the class imbalance problem through meta-learning.
Meta-weight-net: Learning an explicit mapping for sample weighting (NIPS 2019) [Paper][Code] - Explicitly learn a weight function (with an MLP as the function approximator) to reweight the samples in gradient updates of DNN.
Learning Data Manipulation for Augmentation and Weighting (NIPS 2019) [Paper][Code]
Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks (ICLR 2020) [Paper][Code]
MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler (NeurIPS 2020) [Paper][Code][Video]

NOTE: meta-learning-powered ensemble learning

2.5.6 Representation Learning

Learning deep representation for imbalanced classification (CVPR 2016, 220+ citations) [Paper]
Supervised Class Distribution Learning for GANs-Based Imbalanced Classification (ICDM 2019) [Paper]
Decoupling Representation and Classifier for Long-tailed Recognition (ICLR 2020) [Paper][Code]

NOTE: interesting findings on representation learning and classifier learning
Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer (NeurIPS 2021) [Paper]
Tailoring Self-Supervision for Supervised Learning (ECCV 2022) [Paper][Code]

2.5.7 Posterior Recalibration

Posterior Re-calibration for Imbalanced Datasets (NeurIPS 2020) [Paper][Code]
Long-tail learning via logit adjustment (ICLR 2021) [Paper][Code]

2.5.8 Semi/Self-supervised Learning

Rethinking the Value of Labels for Improving Class-Imbalanced Learning (NeurIPS 2020) [Paper][Code][Video]

NOTE: semi-supervised training / self-supervised pre-training helps imbalance learning
Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning (NeurIPS 2020) [Paper][Code]
ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learning (NeurIPS 2021) [Paper][Code]
Improving Contrastive Learning on Imbalanced Data via Open-World Sampling (NeurIPS 2021) [Paper]
DASO: Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning (CVPR 2022) [Paper][Code]

2.5.9 Curriculum Learning

Dynamic Curriculum Learning for Imbalanced Data Classification (ICCV 2019) [Paper]

2.5.10 Two-phase Training

Brain tumor segmentation with deep neural networks (2017, 1200+ citations) [Paper][Code (unofficial)]

Pre-training on balanced dataset, fine-tuning the last output layer before softmax on the original, imbalanced data.

2.5.11 Network Architecture

BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition (CVPR 2020) [Paper][Code]
Class-Imbalanced Deep Learning via a Class-Balanced Ensemble (TNNLS 2021) [Paper]

2.5.12 Deep Generative Model

Deep Generative Model for Robust Imbalance Classification (CVPR 2020) [Paper]

2.5.13 Imbalanced Regression

Delving into Deep Imbalanced Regression (ICML 2021) [Paper][Code][Video]
Density-based weighting for imbalanced regression (Machine Learning [J], 2021) [Paper][Code]

2.5.14 Data Augmentation

Minority-Oriented Vicinity Expansion with Attentive Aggregation for Video Long-Tailed Recognition (AAAI 2023) [Paper][Code]

3. Miscellaneous

3.1 Datasets

imbalanced-learn datasets

This collection of datasets is from imblearn.datasets.fetch_datasets.

ID	Name	Repository & Target	Ratio	#S	#F
1	ecoli	UCI, target: imU	8.6:1	336	7
2	optical_digits	UCI, target: 8	9.1:1	5,620	64
3	satimage	UCI, target: 4	9.3:1	6,435	36
4	pen_digits	UCI, target: 5	9.4:1	10,992	16
5	abalone	UCI, target: 7	9.7:1	4,177	10
6	sick_euthyroid	UCI, target: sick euthyroid	9.8:1	3,163	42
7	spectrometer	UCI, target: > =44	11:1	531	93
8	car_eval_34	UCI, target: good, v good	12:1	1,728	21
9	isolet	UCI, target: A, B	12:1	7,797	617
10	us_crime	UCI, target: >0.65	12:1	1,994	100
11	yeast_ml8	LIBSVM, target: 8	13:1	2,417	103
12	scene	LIBSVM, target: >one label	13:1	2,407	294
13	libras_move	UCI, target: 1	14:1	360	90
14	thyroid_sick	UCI, target: sick	15:1	3,772	52
15	coil_2000	KDD, CoIL, target: minority	16:1	9,822	85
16	arrhythmia	UCI, target: 06	17:1	452	278
17	solar_flare_m0	UCI, target: M->0	19:1	1,389	32
18	oil	UCI, target: minority	22:1	937	49
19	car_eval_4	UCI, target: vgood	26:1	1,728	21
20	wine_quality	UCI, wine, target: <=4	26:1	4,898	11
21	letter_img	UCI, target: Z	26:1	20,000	16
22	yeast_me2	UCI, target: ME2	28:1	1,484	8
23	webpage	LIBSVM, w7a, target: minority	33:1	34,780	300
24	ozone_level	UCI, ozone, data	34:1	2,536	72
25	mammography	UCI, target: minority	42:1	11,183	6
26	protein_homo	KDD CUP 2004, minority	111:1	145,751	74
27	abalone_19	UCI, target: 19	130:1	4,177	10

Imbalanced Databases

Link: https://github.com/gykovacs/mldb

3.2 Github Repositories

3.2.1 Algorithms & Utilities & Jupyter Notebooks

imbalanced-algorithms - Python-based implementations of algorithms for learning on imbalanced data.
imbalanced-dataset-sampler - A (PyTorch) imbalanced dataset sampler for oversampling low frequent classes and undersampling high frequent ones.
class_imbalance - Jupyter Notebook presentation for class imbalance in binary classification.
Multi-class-with-imbalanced-dataset-classification - Perform multi-class classification on imbalanced 20-news-group dataset.
Advanced Machine Learning with scikit-learn: Imbalanced classification and text data - Different approaches to feature selection, and resampling methods for imbalanced data.