Learning Trajectory Dependencies for Human Motion Prediction
This is the code for the paper
Wei Mao, Miaomiao Liu, Mathieu Salzmann, Hongdong Li. Learning Trajectory Dependencies for Human Motion Prediction. In ICCV 19.
Dependencies
- cuda 9.0
- Python 3.6
- Pytorch 0.3.1.
- progress 1.5
Get the data
Human3.6m in exponential map can be downloaded from here.
CMU mocap was obtained from the repo of ConvSeq2Seq paper.
3DPW from their official website.
Quick demo and visualization
For a quick demo, you can train for a few epochs and visualize the outputs of your model.
To train, run
python main.py --epoch 5 --input_n 10 --output 10 --dct_n 20 --data_dir [Path To Your H36M data]/h3.6m/dataset/Visualize the results of pretrained model for predictions on angle space on H36M dataset.
- change the model path
- then run the command below
python demo.py --input_n 10 --output_n 10 --dct_n 20 --data_dir [Path To Your H36M data]/h3.6m/dataset/Training commands
All the running args are defined in opt.py. We use following commands to train on different datasets and representations. To train on angle space,
python main.py --data_dir "[Path To Your H36M data]/h3.6m/dataset/" --input_n 10 --output_n 10 --dct_n 20 --exp [where to save the log file]python main_cmu.py --data_dir_cmu "[Path To Your CMU data]/cmu_mocap/" --input_n 10 --output_n 25 --dct_n 35 --exp [where to save the log file]python main_3dpw.py --data_dir_3dpw "[Path To Your 3DPW data]/3DPW/sequenceFiles/" --input_n 10 --output_n 30 --dct_n 40 --exp [where to save the log file]To train on 3D space,
python3 main_3d.py --data_dir "[Path To Your H36M data]/h3.6m/dataset/" --input_n 10 --output_n 10 --dct_n 15 --exp [where to save the log file]python main_cmu_3d.py --data_dir_cmu "[Path To Your CMU data]/cmu_mocap/" --input_n 10 --output_n 25 --dct_n 30 --exp [where to save the log file]python main_3dpw_3d.py --data_dir_3dpw "[Path To Your 3DPW data]/3DPW/sequenceFiles/" --input_n 10 --output_n 30 --dct_n 35 --exp [where to save the log file]Results
We re-run our code 2 more times under different setups and the overall average results at different time are reported below.
- Human3.6-short-term prediction on angle space (top) and 3D coordinate (bottom)
| 80ms | 160ms | 320ms | 400ms | |
|---|---|---|---|---|
| pre-trained | 0.27 | 0.51 | 0.83 | 0.95 |
| test_run_1 | 0.28 | 0.52 | 0.84 | 0.96 |
| test_run_2 | 0.28 | 0.52 | 0.84 | 0.96 |
| ---------------- | ------ | ------ | ------ | ------ |
| pre-trained | 12.1 | 25.0 | 51.0 | 61.3 |
| test_run_1 | 12.1 | 24.6 | 50.4 | 61.1 |
| test_run_2 | 12.1 | 24.8 | 50.5 | 61.2 |
- Human3.6-long-term prediction
| 560ms | 1000ms | |
|---|---|---|
| pre-trained | 0.90 | 1.27 |
| test_run_1 | 0.91 | 1.25 |
| test_run_2 | 0.92 | 1.27 |
| ------------- | -------- | ------ |
| pre-trained | 50.4 | 71.0 |
| test_run_1 | 51.2 | 71.6 |
| test_run_2 | 51.6 | 70.9 |
- CMU-mocap
| 80ms | 160ms | 320ms | 400ms | 1000ms | |
|---|---|---|---|---|---|
| pre-trained | 0.25 | 0.39 | 0.68 | 0.79 | 1.33 |
| test_run_1 | 0.26 | 0.41 | 0.72 | 0.84 | 1.35 |
| test_run_2 | 0.26 | 0.41 | 0.71 | 0.83 | 1.38 |
| ------------- | ------ | ------- | ------- | ------- | -------- |
| pre-trained | 11.5 | 20.4 | 37.8 | 46.8 | 96.5 |
| test_run_1 | 11.3 | 19.8 | 36.9 | 45.5 | 92.7 |
| test_run_2 | 11.3 | 19.7 | 37.2 | 46.0 | 94.0 |
- 3DPW
| 200ms | 400ms | 600ms | 800ms | 1000ms | |
|---|---|---|---|---|---|
| pre-trained | 0.64 | 0.95 | 1.12 | 1.22 | 1.27 |
| test_run_1 | 0.64 | 0.97 | 1.12 | 1.22 | 1.28 |
| test_run_2 | 0.64 | 0.95 | 1.11 | 1.21 | 1.27 |
| ------------- | ------- | ------- | ------- | ------- | -------- |
| pre-trained | 35.6 | 67.8 | 90.6 | 106.9 | 117.8 |
| test_run_1 | 36.7 | 69.6 | 90.8 | 105.0 | 115.3 |
| test_run_2 | 35.8 | 69.1 | 93.2 | 110.9 | 121.7 |
Citing
If you use our code, please cite our work
@inproceedings{wei2019motion,
title={Learning Trajectory Dependencies for Human Motion Prediction},
author={Wei, Mao and Miaomiao, Liu and Mathieu, Salzemann and Hongdong, Li},
booktitle={ICCV},
year={2019}
}
Acknowledgments
Some of our evaluation code and data process code was adapted/ported from Residual Sup. RNN by Julieta. The overall code framework (dataloading, training, testing etc.) is adapted from 3d-pose-baseline.
Licence
MIT