Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation
Shengju Qian, Keqiang Sun, Wayne Wu, Chen Qian, and Jiaya Jia
International Conference on Computer Vision (ICCV), 2019
This repository contains the implementation of our ICCV 2019 paper, 'Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation'. The proposed method investigates how to boost the performance of facial landmark detector in limited training resources, using synthetic data in a style-translation manner.
Results(NME/%) on WFLW dataset
| Method | Fullset | Pose | Expression | Illumination | Make-up | Occlusion | Blur |
|---|---|---|---|---|---|---|---|
| DVLN | 6.08 | 11.54 | 6.78 | 5.73 | 5.98 | 7.33 | 6.88 |
| LAB | 5.27 | 10.24 | 5.51 | 5.23 | 5.15 | 6.79 | 6.32 |
| SAN | 5.22 | 10.39 | 5.71 | 5.19 | 5.49 | 6.83 | 5.80 |
| WING | 5.11 | 8.75 | 5.36 | 4.93 | 5.41 | 6.37 | 5.81 |
| ResNet-18 | 6.09 | 10.76 | 6.97 | 5.83 | 6.19 | 7.15 | 6.67 |
| AVS(ResNet-18) | 5.25 | 9.10 | 5.83 | 4.93 | 5.47 | 6.26 | 5.86 |
| AVS(LAB) | 4.76 | 8.21 | 5.14 | 4.51 | 5.00 | 5.76 | 5.43 |
| AVS(SAN) | 4.39 | 8.42 | 4.68 | 4.24 | 4.37 | 5.60 | 4.86 |
We provide a synthetic WFLW dataset generated by this method here Google Drive or Baidu Drive.
In the provided dataset, each image is augmented with eight different styles in the original WFLW dataset. You may download from this link and are able to produce similar to re-produce the results reported in our paper.
For example, you can train a SAN(CVPR 2018) model using our script.
Avaliable Implementation
Facial Landmark Detection(SAN)
The instruction of tensorflow implementation is provided in the following:
Prerequisites and Installation
- Python 3.6
- Tensorflow 1.2.1
- CUDA 8.0 and CuDNN 5.1
Getting Started
Clone this repository:
git clone [email protected]:TheSouthFrog/stylealign.git
cd stylealignInstall dependencies using Anaconda:
conda create -n stylealign python=3.6
source activate stylealign
pip install -r requirements.txtNote this implementation, we use variational u-net as network structure as it produces sharper results.
As it uses a deprecated API of data dependent initialization which requires legacy version of CUDA 8.0 and CuDNN 5.1 without backward compatibility.
Prepare Dataset
- You may the original cropped WFLW dataset here Google Drive or Baidu Drive.
- Unzip downloaded files. Remember to change the data_dir and image_list to your image directory and annotation file, respectively.
Running the code:
Assume that you have installed multiple versions of CUDA and CuDNN. By switching the environment to run our code, you need to find out the directory where you have installed the right version and run(for example):
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64Then run:
CUDA_VISIBLE_DEVICES=gpu_id python main.py --config config/exp_0001.yamlGenerate style-augmented samples
CUDA_VISIBLE_DEVICES=gpu_id python main.py --checkpoint XXXXXX --config config/test_0001.yamlFor acceleration, we use the transfer pass to perform testing, where every image in a batch(batch size: n) outputs n samples under different styles in this batch(n2 in single pass).
gen_landmark.py is used to generate seperated images and corresponding labels of synthetic images according to transfer images and given order in original annotation files.
--label_file: The annotation file that used for testing.--checkpoint_iter: Trained Iteration of the testing checkpoint.--batch_size: Testing batch size.--gen_dirand--out_dir: Testing directory and Output directory.
Contact
If you have any questions, please feel free to contact the authors. Shengju Qian [email protected]
Citation
If you use our code, please consider citing our paper:
@inproceedings{qian2019aggregation,
title={Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation},
author={Qian, Shengju and Sun, Keqiang and Wu, Wayne and Qian, Chen and Jia, Jiaya},
journal={ICCV},
year={2019}
}