This project is Pose Estimation on iOS with TensorFlow Lite.
If you are interested in iOS + Machine Learning, visit here you can see various DEMOs.
| 2D pose estimation in real-time | 3D pose estimation |
|---|---|
 |
 |
Features
- Support 2D pose estimaiton TFLite models
- Real-time demo with Metal framwork
- Photo album demo
- Support 3D pose estimation TFLite model
- Real-time demo with Metal framwork (but realtime model is not ready yet)
- Real-time pose matching demo
- Photo album demo
- Render the result keypoints of 2D pose estimation in 2D demo page
- Render the result keypoints of 3D pose estimation with SceneKit
- Render the heatmaps of 2D pose estimation output
- Part Confidence Maps for typical heatmap based models
- Part Affinity Fields for OpenPose (2D multi-person)
- Implemented pose-matching with cosine similiarity in 3D pose demo
- Implemented to fix the shoulders' keypoints in 3D pose estimation to pre-process for pose-matching
Models
Source Images
| Name | gif | img-0 | img-1 | img-2 |
|---|---|---|---|---|
| Source | - |  |
 |
 |
Joint Samples
| Model Names | gif | img-0 | img-1 | img-2 |
|---|---|---|---|---|
| PoseNet | - |  |
 |
 |
| PEFM CPM | - |  |
 |
 |
| PEFM Hourglass | - |  |
 |
 |
| OpenPose (multi-person) | - |  |
 |
 |
Heatmap-ConfidenceMap Samples
| Model Names | gif | img-0 | img-1 | img-2 |
|---|---|---|---|---|
| PoseNet | - | - | - | - |
| PEFM CPM | - |  |
 |
 |
| PEFM Hourglass | - |  |
 |
 |
| OpenPose (multi-person) | - |  |
 |
 |
Heatmap-PAF Samples
| Model Names | gif | img-0 | img-1 | img-2 |
|---|---|---|---|---|
| OpenPose (PAF x) | - |  |
 |
 |
| OpenPose (PAF y) | - |  |
 |
 |
Meta Data
2D
| Name | Size | Inference Time on iPhone11Pro |
Post-process Time on iPhone11Pro |
PCKh-0.5 | multi person vs single person |
Model Source | Paper | tflite download |
|
|---|---|---|---|---|---|---|---|---|---|
| PoseNet | 13.3 MB | - | - | - | single | tensorflow/tensorflow | PersonLab | ||
| PEFM CPM | 2.4 MB | - | - | - | single | edvardHua/PoseEstimationForMobile | Convolutional Pose Machines | ||
| PEFM Hourglass v1 | 1.8 MB | - | - | - | single | edvardHua/PoseEstimationForMobile | Stacked Hourglass Networks | ||
| PEFM Hourglass v2 | 1.7 MB | - | - | - | single | edvardHua/PoseEstimationForMobile | Stacked Hourglass Networks | ||
| OpenPose | 7.8 MB | - | - | - | multi | ildoonet/tf-pose-estimation | OpenPose | ||
| AlphaPose | - | - | - | - | single | osmr/imgclsmob | RMPE | ||
| SelecSLS | - | - | - | - | single | osmr/imgclsmob | - | ||
| IBPPose | - | - | - | - | single | osmr/imgclsmob | - | ||
| Lightweight OpenPose | - | - | - | - | single | osmr/imgclsmob | OpenPose |
3D
| Name | Size | Inference Time on iPhone11Pro |
Post-process Time on iPhone11Pro |
(metric) | Model Source | Paper | tflite download |
|
|---|---|---|---|---|---|---|---|---|
| Baseline3DPose | 137.1 MB | 347 ms | 79 ms | - | mks0601/3DMPPE_POSENET_RELEASE | Baseline3D | download | |
| LiteBaseline3DPose | 16.6 MB | 116 ms (cpu only) |
19 ms (cpu only) |
SangbumChoi/MobileHumanPose | MHP | download |
Requirements
- Xcode 11.3+
- iOS 11.0+
- Swift 5
- CocoaPods
gem install cocoapodsBuild & Run
- Install dependencies with cocoapods
cd ~/{PROJECT_PATH}
pod install- Open the
PoseEstimation-TFLiteSwift.xcworkspaceand run the project
See also
- motlabs/awesome-ml-demos-with-ios
- TensorFlow Lite or Tensorflow models provided by:
- CPM and Hourglass model provided by edvardHua/PoseEstimationForMobile
- PoseNet model provided by tensorflow/examples
- OpenPose model provided by ildoonet/tf-pose-estimation
- Various model provided by osmr/imgclsmob
- 3DMPPE PoseNet model provided by mks0601/3DMPPE_POSENET_RELEASE
- Pose estimation with Core ML - tucan9389/PoseEstimation-CoreML
License
This repository is licensed under Apache 2.0. Full license text is available in LICENSE.