Dual Variational Generation for Low Shot HFR

A PyTorch code of paper Dual Variational Generation for Low Shot Heterogeneous Face Recognition, written with Xiang Wu.

News

The extension version of DVG is published in IEEE TPAMI 2021 (DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition), and its code is released in https://github.com/BradyFU/DVG-Face. The newly released extension version has more powerful performances than this version.

Prerequisites

• Python 2.7
• Pytorch 0.4.1 & torchvision 0.2.1

Train the generator

• Download LightCNN-29 model (Google Drive) pretrained on the MS-Celeb-1M dataset.
• Train the generator:

sh run_train_generator.sh

• Note that this is a simplified version of our original code:
  1. The diversity loss and the adversarial loss in the paper are removed.
  2. The distribution alignment loss is replaced by a Maximum Mean Discrepancy (MMD) loss.
• The generated results during training will be saved in ./results.

Generate images from noise

• Use the trained generator to sample 100,000 paired heterogeneous data:

Python val.py --pre_model './model/netG_model_epoch_50_iter_0.pth'

• The generated fake NIR and VIS images will be saved in ./fake_images/nir_noise and ./fake_images/vis_noise, respectively.

Train the recognition model LightCNN-29

• Use the real data and the generated fake data to train lightcnn:

sh run_train_lightcnn.sh

Performance

The performance on the 1-fold of CASIA NIR-VIS 2.0 dataset after running the above code:

Citation

If you use our code for your research, please cite the following paper:

@article{fu2021dvg,
  title={DVG-face: Dual variational generation for heterogeneous face recognition},
  author={Fu, Chaoyou and Wu, Xiang and Hu, Yibo and Huang, Huaibo and He, Ran},
  journal={IEEE TPAMI},
  year={2021}
}

@inproceedings{fu2019dual,
  title={Dual Variational Generation for Low-Shot Heterogeneous Face Recognition},
  author={Fu, Chaoyou and Wu, Xiang and Hu, Yibo and Huang, Huaibo and He, Ran},
  booktitle={NeurIPS},
  year={2019}
}