An unofficial PyTorch implementation of MPIIGaze and MPIIFaceGaze

Here is a demo program. See also this repo.

Requirements

• Linux (Tested on Ubuntu only)
• Python >= 3.7

pip install -r requirements.txt

Download the dataset and preprocess it

MPIIGaze

bash scripts/download_mpiigaze_dataset.sh
python tools/preprocess_mpiigaze.py --dataset datasets/MPIIGaze -o datasets/

MPIIFaceGaze

bash scripts/download_mpiifacegaze_dataset.sh
python tools/preprocess_mpiifacegaze.py --dataset datasets/MPIIFaceGaze_normalized -o datasets/

Usage

This repository uses YACS for configuration management. Default parameters are specified in gaze_estimation/config/defaults.py (which is not supposed to be modified directly). You can overwrite those default parameters using a YAML file like configs/mpiigaze/lenet_train.yaml.

Training and Evaluation

By running the following code, you can train a model using all the data except the person with ID 0, and run test on that person.

python train.py --config configs/mpiigaze/lenet_train.yaml
python evaluate.py --config configs/mpiigaze/lenet_eval.yaml

Using scripts/run_all_mpiigaze_lenet.sh and scripts/run_all_mpiigaze_resnet_preact.sh, you can run all training and evaluation for LeNet and ResNet-8 with default parameters.

Results

MPIIGaze

The training time is the value when using GTX 1080Ti.

MPIIFaceGaze

The training time is the value when using GTX 1080Ti.

Demo

This demo program runs gaze estimation on the video from a webcam.

1. Download the dlib pretrained model for landmark detection.

   bash scripts/download_dlib_model.sh

2. Calibrate the camera.

   Save the calibration result in the same format as the sample file data/calib/sample_params.yaml.

3. Run demo.

   Specify the model path and the path of the camera calibration results in the configuration file as in configs/demo_mpiigaze_resnet.yaml.

   python demo.py --config configs/demo_mpiigaze_resnet.yaml

Related repos

• https://github.com/hysts/pl_gaze_estimation
• https://github.com/hysts/pytorch_mpiigaze_demo

References

• Zhang, Xucong, Yusuke Sugano, Mario Fritz, and Andreas Bulling. "Appearance-based Gaze Estimation in the Wild." Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. arXiv:1504.02863, Project Page
• Zhang, Xucong, Yusuke Sugano, Mario Fritz, and Andreas Bulling. "It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation." Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW), 2017. arXiv:1611.08860, Project Page
• Zhang, Xucong, Yusuke Sugano, Mario Fritz, and Andreas Bulling. "MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation." IEEE transactions on pattern analysis and machine intelligence 41 (2017). arXiv:1711.09017
• Zhang, Xucong, Yusuke Sugano, and Andreas Bulling. "Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications." Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), 2019. arXiv, code