DeepStream Human Pose Estimation

Human pose estimation is the computer vision task of estimating the configuration (‘the pose’) of the human body by localizing certain key points on a body within a video or a photo. The following application serves as a reference to deploy custom pose estimation models with DeepStream 5.0 using the TRTPose project as an example.

A detailed deep-dive NVIDIA Developer blog is available here.

Input Video Source		Output Video

Prerequisites

You will need

1. DeepStreamSDK 5.0
2. CUDA 10.2
3. TensorRT 7.x

Getting Started:

To get started, please follow these steps.

1. Install DeepStream on your platform, verify it is working by running deepstream-app.
2. Clone the repository preferably in $DEEPSTREAM_DIR/sources/apps/sample_apps.
3. Download the TRTPose model, convert it to ONNX using this export utility, and set its location in the DeepStream configuration file.
4. Replace the OSD binaries (x86 or Jetson) in $DEEPSTREAM_DIR/libs with the ones provided in this repository under bin/. Please note that these are not inter-compatible across platforms.
5. Compile the program

 $ cd deepstream-pose-estimation/
 $ sudo make
 $ sudo ./deepstream-pose-estimation-app <file-uri> <output-path>

5. The final output is stored in 'output-path' as Pose_Estimation.mp4

NOTE: If you do not already have a .trt engine generated from the ONNX model you provided to DeepStream, an engine will be created on the first run of the application. Depending upon the system you’re using, this may take anywhere from 4 to 10 minutes.

For any issues or questions, please feel free to make a new post on the DeepStreamSDK forums.

References

Cao, Zhe, et al. "Realtime multi-person 2d pose estimation using part affinity fields." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.

Xiao, Bin, Haiping Wu, and Yichen Wei. "Simple baselines for human pose estimation and tracking." Proceedings of the European Conference on Computer Vision (ECCV). 2018.