Out-of-distribution Detection with Deep Nearest Neighbors

This is the source code for ICML 2022 paper Out-of-distribution Detection with Deep Nearest Neighbors by Yiyou Sun, Yifei Ming, Xiaojin Zhu and Yixuan Li.

Usage

1. Dataset Preparation for Large-scale Experiment

In-distribution dataset

Please download ImageNet-1k and place the training data and validation data in ./datasets/imagenet/train and ./datasets/imagenet/val, respectively.

Out-of-distribution dataset

We have curated 4 OOD datasets from iNaturalist, SUN, Places, and Textures, and de-duplicated concepts overlapped with ImageNet-1k.

For iNaturalist, SUN, and Places, we have sampled 10,000 images from the selected concepts for each dataset, which can be download via the following links:

wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/iNaturalist.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/SUN.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/Places.tar.gz

For Textures, we use the entire dataset, which can be downloaded from their original website.

Please put all downloaded OOD datasets into ./datasets/ood_data.

2. Dataset Preparation for CIFAR Experiment

In-distribution dataset

The downloading process will start immediately upon running.

Out-of-distribution dataset

We provide links and instructions to download each dataset:

• SVHN: download it and place it in the folder of datasets/ood_data/svhn. Then run python select_svhn_data.py to generate test subset.
• Textures: download it and place it in the folder of datasets/ood_data/dtd.
• Places365: download it and place it in the folder of datasets/ood_data/places365/test_subset. We randomly sample 10,000 images from the original test dataset.
• LSUN: download it and place it in the folder of datasets/ood_data/LSUN.
• iSUN: download it and place it in the folder of datasets/ood_data/iSUN.
• LSUN_fix: download it and place it in the folder of datasets/ood_data/LSUN_fix.
• ImageNet_fix: download it and place it in the folder of datasets/ood_data/ImageNet_fix.
• ImageNet_resize: download it and place it in the folder of datasets/ood_data/Imagenet_resize.

3. Pre-trained model

Please download Pre-trained models and place in the ./checkpoints folder.

Preliminaries

It is tested under Ubuntu Linux 20.04 and Python 3.8 environment, and requries some packages to be installed:

• PyTorch
• scipy
• numpy
• sklearn
• faiss
• pytorch-vit
• ylib (Manually download and copy to the current folder)

Demo

1. Demo code for ImageNet Experiment

Run ./demo_imagenet.sh.

2. Demo code for CIFAR Experiment

Run ./demo_cifar.sh.

Citation

If you use our codebase, please cite our work:

@article{sun2022knnood,
  title={Out-of-distribution Detection with Deep Nearest Neighbors},
  author={Sun, Yiyou and Ming, Yifei and Zhu, Xiaojin and Li, Yixuan},
  journal={ICML},
  year={2022}
}