ENSTA-U2IS/awesome-uncertainty-deeplearning ENSTA-U2IS/awesome-uncertainty-deeplearning

  • Stars
    star
    304
  • Rank 137,274 (Top 3 %)
  • Language
  • License
    MIT License
  • Created almost 3 years ago
  • Updated about 1 year ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
ENSTA-U2IS/awesome-uncertainty-deeplearning
github

ENSTA-U2IS

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

This repository contains a collection of surveys, datasets, papers, and codes, for predictive uncertainty estimation in deep learning models.

Awesome Uncertainty in Deep learning

MIT License Awesome

This repo is a collection of AWESOME papers, codes, books, and blogs about Uncertainty and Deep learning. Feel free to star and fork.

If you think that we miss a paper, or if you have any ideas for improvements, please send a message on the corresponding GitHub discussions.

You may also send an email at: gianni dot franchi at ensta-paris dot fr with "[Awesome Uncertainty]" as subject. Tell us where the paper was published and when, and send us GitHub and ArXiv links if they are available.

Table of Contents

Papers

Surveys

Conference

  • A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications [AISafety Workshop 2020]

Journal

Arxiv

  • A Survey on Uncertainty Reasoning and Quantification for Decision Making: Belief Theory Meets Deep Learning [arXiv2022]
  • A survey of uncertainty in deep neural networks [arXiv2021] - [GitHub]
  • A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation [arXiv2021]

Theory

Conference

  • Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs [ICML2023] - [PyTorch]
  • On Second-Order Scoring Rules for Epistemic Uncertainty Quantification [ICML2023]
  • Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning [ICLR2023]
  • Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? [ICLR2023]
  • Top-label calibration and multiclass-to-binary reductions [ICLR2022]
  • Bayesian Model Selection, the Marginal Likelihood, and Generalization [ICML2022]
  • Neural Variational Gradient Descent [AABI2022]
  • Repulsive Deep Ensembles are Bayesian [NeurIPS2021] - [PyTorch]
  • Bayesian Optimization with High-Dimensional Outputs [NeurIPS2021]
  • Residual Pathway Priors for Soft Equivariance Constraints [NeurIPS2021]
  • Dangers of Bayesian Model Averaging under Covariate Shift [NeurIPS2021] - [TensorFlow]
  • With malice towards none: Assessing uncertainty via equalized coverage [AIES 2021]
  • A Mathematical Analysis of Learning Loss for Active Learning in Regression [CVPR Workshop2021]
  • Uncertainty in Gradient Boosting via Ensembles [ICLR2021] - [PyTorch]
  • Deep Convolutional Networks as shallow Gaussian Processes [ICLR2019]
  • On the accuracy of influence functions for measuring group effects [NeurIPS2018]
  • To Trust Or Not To Trust A Classifier [NeurIPS2018] - [Python]
  • Understanding Measures of Uncertainty for Adversarial Example Detection [UAI2018]

Journal

Arxiv

  • Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping [arXiv2022]
  • Efficient Gaussian Neural Processes for Regression [arXiv2021]
  • Dense Uncertainty Estimation [arXiv2021] - [PyTorch]
  • A higher-order swiss army infinitesimal jackknife [arXiv2019]

Bayesian-Methods

Conference

  • Gradient-based Uncertainty Attribution for Explainable Bayesian Deep Learning [CVPR2023]
  • Robustness to corruption in pre-trained Bayesian neural networks [ICLR2023]
  • Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture [NeurIPS2022]
  • Activation-level uncertainty in deep neural networks [ICLR2021]
  • On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks [UAI2021]
  • Learnable uncertainty under Laplace approximations [UAI2021]
  • Bayesian Deep Learning and a Probabilistic Perspective of Generalization [NeurIPS2020]
  • On Batch Normalisation for Approximate Bayesian Inference [AABI2021]
  • How Good is the Bayes Posterior in Deep Neural Networks Really? [ICML2020]
  • Bayesian Uncertainty Estimation for Batch Normalized Deep Networks [ICML2020]
  • Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors [ICML2020] - [TensorFlow]
  • Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks [ICML2020]
  • TRADI: Tracking deep neural network weight distributions for uncertainty estimation [ECCV2020] - [PyTorch]
  • A Simple Baseline for Bayesian Uncertainty in Deep Learning [NeurIPS2019] - [PyTorch]
  • Lightweight Probabilistic Deep Networks [CVPR2018] - [PyTorch]
  • A Scalable Laplace Approximation for Neural Networks [ICLR2018] - [Theano]
  • Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning [ICML2018]
  • Weight Uncertainty in Neural Networks [ICML2015]

Journal

  • Bayesian modeling of uncertainty in low-level vision [IJCV1990]

Arxiv

  • Density Uncertainty Layers for Reliable Uncertainty Estimation [arXiv2023]
  • Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantification [arXiv2020] - [PyTorch]
  • Bayesian Neural Networks with Soft Evidence [arXiv2020] - [PyTorch]
  • Bayesian neural network via stochastic gradient descent [arXiv2020]

Ensemble-Methods

Conference

  • Weighted Ensemble Self-Supervised Learning [ICLR2023]
  • Agree to Disagree: Diversity through Disagreement for Better Transferability [ICLR2023] - [PyTorch]
  • Packed-Ensembles for Efficient Uncertainty Estimation [ICLR2023] - [PyTorch]
  • Normalizing Flow Ensembles for Rich Aleatoric and Epistemic Uncertainty Modeling [AAAI2023]
  • Deep Ensembles Work, But Are They Necessary? [NeurIPS2022]
  • FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation [NeurIPS2022]
  • Prune and Tune Ensembles: Low-Cost Ensemble Learning With Sparse Independent Subnetworks [AAAI2022]
  • Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity [ICLR2022] - [PyTorch]
  • Robustness via Cross-Domain Ensembles [ICCV2021] - [PyTorch]
  • Masksembles for Uncertainty Estimation [CVPR2021] - [PyTorch/TensorFlow]
  • Uncertainty Quantification and Deep Ensembles [NeurIPS2021]
  • Uncertainty in Gradient Boosting via Ensembles [ICLR2021] - [PyTorch]
  • Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning [ICLR2020] - [PyTorch]
  • Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensembles [AAAI2020]
  • Hyperparameter Ensembles for Robustness and Uncertainty Quantification [NeurIPS2020]
  • Bayesian Deep Ensembles via the Neural Tangent Kernel [NeurIPS2020]
  • BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning [ICLR2020] - [TensorFlow] - [PyTorch]
  • Uncertainty in Neural Networks: Approximately Bayesian Ensembling [AISTATS 2020]
  • Accurate Uncertainty Estimation and Decomposition in Ensemble Learning [NeurIPS2019]
  • Diversity with Cooperation: Ensemble Methods for Few-Shot Classification [ICCV2019]
  • High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach [ICML2018] - [TensorFlow]
  • Simple and scalable predictive uncertainty estimation using deep ensembles [NeurIPS2017]

Journal

  • One Versus all for deep Neural Network for uncertaInty (OVNNI) quantification [IEEE Access2021]

Arxiv

  • On the Usefulness of Deep Ensemble Diversity for Out-of-Distribution Detection [arXiv2022]
  • Deep Ensemble as a Gaussian Process Approximate Posterior [arXiv2022]
  • Sequential Bayesian Neural Subnetwork Ensembles [arXiv2022]
  • Confident Neural Network Regression with Bootstrapped Deep Ensembles [arXiv2022] - [TensorFlow]
  • Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model [arXiv2021]
  • Deep Ensembles: A Loss Landscape Perspective [arXiv2019]

Sampling/Dropout-based-Methods

Conference

  • Efficient Bayesian Uncertainty Estimation for nnU-Net [MICCAI2022]
  • Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate [AAAI2022]
  • Dropout Sampling for Robust Object Detection in Open-Set Conditions [ICRA2018]
  • Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks [MIDL2018]
  • Concrete Dropout [NeurIPS2017]
  • Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning [ICML2016]

Journal

Arxiv

  • SoftDropConnect (SDC) โ€“ Effective and Efficient Quantification of the Network Uncertainty in Deep MR Image Analysis [arXiv2022]
  • Wasserstein Dropout [arXiv2021] - [PyTorch]

Auxiliary-Methods/Learning-loss-distributions

Conference

  • Post-hoc Uncertainty Learning using a Dirichlet Meta-Model [AAAI2023] - [PyTorch]
  • Improving the reliability for confidence estimation [ECCV2022]
  • Gradient-based Uncertainty for Monocular Depth Estimation [ECCV2022] - [PyTorch]
  • BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks [ECCV2022] - [PyTorch]
  • Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model [AAAI2022]
  • Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation [NeurIPS2022]
  • Learning Structured Gaussians to Approximate Deep Ensembles [CVPR2022]
  • Learning Uncertainty For Safety-Oriented Semantic Segmentation In Autonomous Driving [ICIP2022]
  • SLURP: Side Learning Uncertainty for Regression Problems [BMVC2021] - [PyTorch]
  • Learning to Predict Error for MRI Reconstruction [MICCAI2021]
  • Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation [ICCV2021] - [PyTorch]
  • A Mathematical Analysis of Learning Loss for Active Learning in Regression [CVPR Workshop2021]
  • Real-time uncertainty estimation in computer vision via uncertainty-aware distribution distillation [WACV2021]
  • Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel [ICLR2020] - [TensorFlow]
  • Gradients as a Measure of Uncertainty in Neural Networks [ICIP2020]
  • Learning Loss for Test-Time Augmentation [NeurIPS2020]
  • On the uncertainty of self-supervised monocular depth estimation [CVPR2020] - [PyTorch]
  • Addressing failure prediction by learning model confidence [NeurIPS2019] - [PyTorch]
  • Learning loss for active learning [CVPR2019] - [PyTorch] (unofficial codes)
  • Structured Uncertainty Prediction Networks [CVPR2018] - [TensorFlow]
  • Uncertainty estimates and multi-hypotheses networks for optical flow [ECCV2018] - [TensorFlow]
  • Classification uncertainty of deep neural networks based on gradient information [IAPR Workshop2018]
  • What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? [NeurIPS2017]
  • Estimating the Mean and Variance of the Target Probability Distribution [(ICNN94)]

Journal

  • Confidence Estimation via Auxiliary Models [TPAMI2021]

Arxiv

  • Instance-Aware Observer Network for Out-of-Distribution Object Segmentation [arXiv2022]
  • DEUP: Direct Epistemic Uncertainty Prediction [arXiv2020]
  • Learning Confidence for Out-of-Distribution Detection in Neural Networks [arXiv2018]

Data-augmentation/Generation-based-methods

Conference

  • On the Pitfall of Mixup for Uncertainty Calibration [CVPR2023]
  • Diverse, Global and Amortised Counterfactual Explanations for Uncertainty Estimates [AAAI2022]
  • PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures [CVPR2022]
  • Towards efficient feature sharing in MIMO architectures [CVPR Workshop2022]
  • Robust Semantic Segmentation with Superpixel-Mix [BMVC2021] - [PyTorch]
  • MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks [ICCV2021] - [PyTorch]
  • Training independent subnetworks for robust prediction [ICLR2021]
  • Uncertainty-aware GAN with Adaptive Loss for Robust MRI Image Enhancement [ICCV Workshop2021]
  • Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning [ICML2020]
  • Uncertainty-Aware Deep Classifiers using Generative Models [AAAI2020]
  • Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation [ECCV2020] - [PyTorch]
  • Detecting the Unexpected via Image Resynthesis [ICCV2019] - [PyTorch]
  • On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks [NeurIPS2019]
  • Deep Anomaly Detection with Outlier Exposure [ICLR2019]

Arxiv

  • ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inference [arXiv2022]
  • Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [arXiv2021]
  • Quantifying uncertainty with GAN-based priors [arXiv2019]

Dirichlet-networks/Evidential-deep-learning

Conference

  • Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [ICML2023]
  • Exploring and Exploiting Uncertainty for Incomplete Multi-View Classification [CVPR2023]
  • Fast Predictive Uncertainty for Classification with Bayesian Deep Networks [UAI2022] - [PyTorch]
  • An Evidential Neural Network Model for Regression Based on Random Fuzzy Numbers [BELIEF2022]
  • Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions [ICLR2022] - [PyTorch]
  • Improving Evidential Deep Learning via Multi-task Learning [AAAI2022]
  • Trustworthy multimodal regression with mixture of normal-inverse gamma distributions [NeurIPS2021]
  • Misclassification Risk and Uncertainty Quantification in Deep Classifiers [WACV2021]
  • Evaluating robustness of predictive uncertainty estimation: Are Dirichlet-based models reliable? [ICML2021]
  • Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts [NeurIPS2020] - [PyTorch]
  • Being Bayesian about Categorical Probability [ICML2020]
  • Ensemble Distribution Distillation [ICLR2020]
  • Conservative Uncertainty Estimation By Fitting Prior Networks [ICLR2020]
  • Noise Contrastive Priors for Functional Uncertainty [UAI2020]
  • Deep Evidential Regression [NeurIPS2020] - [TensorFlow]
  • Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examples [NeurIPS Workshop2020]
  • Uncertainty on Asynchronous Time Event Prediction [NeurIPS2019] - [TensorFlow]
  • Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness [NeurIPS2019]
  • Quantifying Classification Uncertainty using Regularized Evidential Neural Networks [AAAI FSS2019]
  • Evidential Deep Learning to Quantify Classification Uncertainty [NeurIPS2018] - [PyTorch]
  • Predictive uncertainty estimation via prior networks [NeurIPS2018]

Journal

Arxiv

  • Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [arXiv2023]
  • The Unreasonable Effectiveness of Deep Evidential Regression [arXiv2022]
  • Effective Uncertainty Estimation with Evidential Models for Open-World Recognition [arXiv2022]
  • Multivariate Deep Evidential Regression [arXiv2022]
  • A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation [arXiv2021]
  • Regression Prior Networks [arXiv2020]
  • A Variational Dirichlet Framework for Out-of-Distribution Detection [arXiv2019]
  • Uncertainty estimation in deep learning with application to spoken language assessment[PhDThesis2019]
  • Inhibited softmax for uncertainty estimation in neural networks [arXiv2018].
  • Quantifying Intrinsic Uncertainty in Classification via Deep Dirichlet Mixture Networks [arXiv2018]

Deterministic-Uncertainty-Methods

Conference

  • Deep Deterministic Uncertainty: A Simple Baseline [CVPR2023] - [PyTorch]
  • Training, Architecture, and Prior for Deterministic Uncertainty Methods [ICLR Workshop2023] - [PyTorch]
  • Latent Discriminant deterministic Uncertainty [ECCV2022] - [PyTorch]
  • Improving Deterministic Uncertainty Estimation in Deep Learning for Classification and Regression [CoRR2021]
  • Training normalizing flows with the information bottleneck for competitive generative classification [NeurIPS2020]
  • Simple and principled uncertainty estimation with deterministic deep learning via distance awareness [NeurIPS2020]
  • Uncertainty Estimation Using a Single Deep Deterministic Neural Network [ICML2020] - [PyTorch]
  • Single-Model Uncertainties for Deep Learning [NeurIPS2019] - [PyTorch]
  • Sampling-Free Epistemic Uncertainty Estimation Using Approximated Variance Propagation [ICCV2019] - [PyTorch]

Journal

Arxiv

  • On the Practicality of Deterministic Epistemic Uncertainty [arXiv2021]
  • The Hidden Uncertainty in a Neural Networkโ€™s Activations [arXiv2020]
  • A simple framework for uncertainty in contrastive learning [arXiv2020]
  • Distance-based Confidence Score for Neural Network Classifiers [arXiv2017]

Quantile-Regression/Predicted-Intervals

Conference

Journal

  • Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors [CMAME2022]
  • Exploring uncertainty in regression neural networks for construction of prediction intervals [Neurocomputing2022]

Arxiv

  • Interval Neural Networks: Uncertainty Scores [arXiv2020]
  • Tight Prediction Intervals Using Expanded Interval Minimization [arXiv2018]

Calibration

Conference

  • Beyond calibration: estimating the grouping loss of modern neural networks [ICLR2023]
  • Rethinking Confidence Calibration for Failure Prediction [ECCV2022] - [PyTorch]
  • The Devil is in the Margin: Margin-based Label Smoothing for Network Calibration [CVPR2022] - [PyTorch]
  • Calibrating Deep Neural Networks by Pairwise Constraints [CVPR2022]
  • Top-label calibration and multiclass-to-binary reductions [ICLR2022]
  • Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error [ICML Workshop2021] - [PyTorch]
  • Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification [NeurIPS2021]
  • Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain [AIStats2021]
  • From label smoothing to label relaxation [AAAI2021]
  • Calibrating Deep Neural Networks using Focal Loss [NeurIPS2020] - [PyTorch]
  • Stationary activations for uncertainty calibration in deep learning [NeurIPS2020]
  • Confidence-Aware Learning for Deep Neural Networks [ICML2020] - [PyTorch]
  • Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning [ICML2020]
  • Regularization via structural label smoothing [ICML2020]
  • Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning [MIDL2020] - [PyTorch]
  • Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision [CVPR Workshop2020] - [PyTorch]
  • Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration [NeurIPS2019] - [GitHub]
  • When does label smoothing help? [NeurIPS2019]
  • Verified Uncertainty Calibration [NeurIPS2019]
  • Generalized zero-shot learning with deep calibration network [NeurIPS2018]
  • Measuring Calibration in Deep Learning [CVPR Workshop2019]
  • Accurate Uncertainties for Deep Learning Using Calibrated Regression [ICML2018]
  • On calibration of modern neural networks [ICML2017]
  • On Fairness and Calibration [NeurIPS2017]
  • Obtaining Well Calibrated Probabilities Using Bayesian Binning [AAAI2015]

Journal

Arxiv

  • Towards Understanding Label Smoothing [arXiv2020]
  • An Investigation of how Label Smoothing Affects Generalization [arXiv2020]

Applications

Classification and Semantic-Segmentation

Conference

  • CRISP - Reliable Uncertainty Estimation for Medical Image Segmentation [MICCAI2022]
  • TBraTS: Trusted Brain Tumor Segmentation [MICCAI2022] - [PyTorch]
  • Anytime Dense Prediction with Confidence Adaptivity [ICLR2022] - [PyTorch]
  • Robust Semantic Segmentation with Superpixel-Mix [BMVC2021] - [PyTorch]
  • Classification with Valid and Adaptive Coverage [NeurIPS2020]
  • DEAL: Difficulty-aware Active Learning for Semantic Segmentation [ACCV2020]
  • Human Uncertainty Makes Classification More Robust [ICCV2019]
  • Classification uncertainty of deep neural networks based on gradient information [IAPR Workshop2018]
  • Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation [ICCV2019]
  • Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation [MICCAI2019] - [PyTorch]
  • A Probabilistic U-Net for Segmentation of Ambiguous Images [NeurIPS2018] - [PyTorch]
  • Evidential Deep Learning to Quantify Classification Uncertainty [NeurIPS2018] - [PyTorch]
  • Lightweight Probabilistic Deep Networks [CVPR2018] - [PyTorch]
  • To Trust Or Not To Trust A Classifier [NeurIPS2018]
  • Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding [BMVC2017]

Journal

  • Explainable machine learning in image classification models: An uncertainty quantification perspective." [KnowledgeBased2022]
  • Region-Based Evidential Deep Learning to Quantify Uncertainty and Improve Robustness of Brain Tumor Segmentation [NCA2022]

Arxiv

  • Deep Deterministic Uncertainty for Semantic Segmentation [arXiv2021]
  • Evaluating Bayesian Deep Learning Methods for Semantic Segmentation [arXiv2018]

Regression

Conference

  • Learning the Distribution of Errors in Stereo Matching for Joint Disparity and Uncertainty Estimation [CVPR2023] - [PyTorch]
  • Variational Depth Networks: Uncertainty-Aware Monocular Self-supervised Depth Estimation [ECCV Workshop2022]
  • Uncertainty Quantification in Depth Estimation via Constrained Ordinal Regression [ECCV2022]
  • On Monocular Depth Estimation and Uncertainty Quantification using Classification Approaches for Regression [ICIP2022]
  • Anytime Dense Prediction with Confidence Adaptivity [ICLR2022] - [PyTorch]
  • Learning Structured Gaussians to Approximate Deep Ensembles [CVPR2022]
  • Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate [AAAI2022]
  • Robustness via Cross-Domain Ensembles [ICCV2021] - [PyTorch]
  • SLURP: Side Learning Uncertainty for Regression Problems [BMVC2021] - [PyTorch]
  • Learning to Predict Error for MRI Reconstruction [MICCAI2021]
  • Deep Evidential Regression [NeurIPS2020] - [TensorFlow]
  • Quantifying Point-Prediction Uncertainty in Neural Networks via Residual Estimation with an I/O Kernel [ICLR2020] - [TensorFlow]
  • Well-Calibrated Regression Uncertainty in Medical Imaging with Deep Learning [MIDL2020] - [PyTorch]
  • On the uncertainty of self-supervised monocular depth estimation [CVPR2020] - [PyTorch]
  • Fast Uncertainty Estimation for Deep Learning Based Optical Flow [IROS2020]
  • Inferring Distributions Over Depth from a Single Image [IROS2019] - [TensorFlow]
  • Multi-Task Learning based on Separable Formulation of Depth Estimation and its Uncertainty [CVPR Workshop2019]
  • Lightweight Probabilistic Deep Networks [CVPR2018] - [PyTorch]
  • Uncertainty estimates and multi-hypotheses networks for optical flow [ECCV2018] - [TensorFlow]
  • Accurate Uncertainties for Deep Learning Using Calibrated Regression [ICML2018]
  • Structured Uncertainty Prediction Networks [CVPR2018] - [TensorFlow]

Journal

Arxiv

  • How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts? [arXiv2023] - [PyTorch]
  • UncertaINR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomographaphy [arXiv2022]
  • Efficient Gaussian Neural Processes for Regression [arXiv2021]

Anomaly-detection and Out-of-Distribution-Detection

Conference

  • Uncertainty-Aware Optimal Transport for Semantically Coherent Out-of-Distribution Detection [CVPR2023] - [PyTorch]
  • Modeling the Distributional Uncertainty for Salient Object Detection Models [CVPR2023] - [PyTorch]
  • SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection [CVPR2023] - [PyTorch]
  • How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection? [ICLR2023] - [PyTorch]
  • Can CNNs Be More Robust Than Transformers? [ICLR2023]
  • Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization [ICLR2023]
  • A framework for benchmarking class-out-of-distribution detection and its application to ImageNet [ICLR2023]
  • Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data [ACCV2022]
  • Detecting Misclassification Errors in Neural Networks with a Gaussian Process Model [AAAI2022]
  • VOS: Learning What You Don't Know by Virtual Outlier Synthesis [ICLR2022] - [PyTorch]
  • Anomaly Detection via Reverse Distillation from One-Class Embedding [CVPR2022]
  • Towards Total Recall in Industrial Anomaly Detection [CVPR2022] - [PyTorch]
  • Fully Convolutional Cross-Scale-Flows for Image-based Defect Detection [WACV2022] - [PyTorch]
  • Out-of-Distribution Detection Using Union of 1-Dimensional Subspaces [CVPR2021] - [PyTorch]
  • On the Importance of Gradients for Detecting Distributional Shifts in the Wild [NeurIPS2021]
  • Exploring the Limits of Out-of-Distribution Detection [NeurIPS2021]
  • NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization [ICCV2021]
  • NADS: Neural Architecture Distribution Search for Uncertainty Awareness [ICML2020]
  • Energy-based Out-of-distribution Detection [NeurIPS2020]
  • PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization [ICPR2020] - [PyTorch]
  • Detecting out-of-distribution image without learning from out-of-distribution data. [CVPR2020]
  • Learning Open Set Network with Discriminative Reciprocal Points [ECCV2020]
  • Synthesize then Compare: Detecting Failures and Anomalies for Semantic Segmentation [ECCV2020] - [PyTorch]
  • Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examples [NeurIPS Workshop2020]
  • Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection [ICCV2019] - [PyTorch]
  • Detecting the Unexpected via Image Resynthesis [ICCV2019] - [PyTorch]
  • Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks [ICLR2018]
  • A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks [ICLR2017] - [TensorFlow]

Journal

  • One Versus all for deep Neural Network for uncertaInty (OVNNI) quantification [IEEE Access2021]

Arxiv

  • Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization [arXiv2023] - [PyTorch]
  • A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection [arXiv2021]
  • Generalized out-of-distribution detection: A survey [arXiv2021]
  • Do We Really Need to Learn Representations from In-domain Data for Outlier Detection? [arXiv2021]
  • DATE: Detecting Anomalies in Text via Self-Supervision of Transformers [arXiv2021]
  • Frequentist uncertainty estimates for deep learning [arXiv2018]

Object detection

Conference

  • Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection [CVPR2023]
  • Estimating and Evaluating Regression Predictive Uncertainty in Deep Object Detectors [ICLR2021]
  • Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving [ICCV2019] - [CUDA] - [PyTorch] - [Keras]

Domain adaptation

Conference

  • Guiding Pseudo-labels with Uncertainty Estimation for Source-free Unsupervised Domain Adaptation [CVPR2023] - [PyTorch]
  • Uncertainty-guided Source-free Domain Adaptation [ECCV2022] - [PyTorch]

Natural Language Processing

  • R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents [ICML2023] - [GitHub]

Datasets and Benchmarks

  • SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation [CVPR2022]
  • MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks [BMVC2022] - [PyTorch]
  • ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding [ICCV2021]
  • The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection [IJCV2021]
  • SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation [NeurIPS2021]
  • Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning [arXiv2021] - [TensorFlow]
  • Curriculum Model Adaptation with Synthetic and Real Data for Semantic Foggy Scene Understanding [IJCV2020]
  • Benchmarking the Robustness of Semantic Segmentation Models [CVPR2020]
  • Fishyscapes: A Benchmark for Safe Semantic Segmentation in Autonomous Driving [ICCV Workshop2019]
  • Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming [NeurIPS Workshop2019] - [GitHub]
  • Semantic Foggy Scene Understanding with Synthetic Data [IJCV2018]
  • Lost and Found: Detecting Small Road Hazards for Self-Driving Vehicles [IROS2016]

Libraries

Lectures and tutorials

Books

  • The "Probabilistic Machine-Learning" book series by Kevin Murphy [Book]

Other resources

Awesome conformal prediction [GitHub]

Uncertainty Quantification in Deep Learning [GitHub]

Awesome LLM Uncertainty Reliability Robustness [GitHub]

More Repositories

1

torch-uncertainty

Uncertainty quantification for deep learning models in PyTorch ๐ŸŒฑ
Python
108
star
2

LDU

Latent Discriminant deterministic Uncertainty [ECCV2022]
Python
33
star
3

MUAD-Dataset

MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks [BMVC 2022]
Python
21
star
4

MR_perceptual

A Study of Deep Perceptual Metrics for Image quality Assessment
Python
10
star
5

DeepLabV3Plus-MUAD-Pytorch

Deeplab V3+ on the MUAD dataset
Python
8
star
6

NAO_morpho

Python
4
star
7

infraParis

Multimodal & infrared automotive dataset
JavaScript
1
star