Join our Mailing List! Sign up to stay up-to-date on our latest feature release, community events, blog posts, and more!
The Perception package provides a toolkit for generating large-scale synthetic datasets for computer vision training and validation.
Getting Started with the Perception Package
Ideal for those new to either Unity, the Perception package, or synthetic data in general. Detailed instructions covering all the important steps: installing the Unity Editor, creating your first synthetic data generation project, adding domain randomization, visualizing, and analyzing your generated datasets. No prior experience required.
Quick Start Guide
Instructions for adding the Perception package to your existing project. Recommended for users already familiar with Unity and the Perception package. If this is the first time exploring our toolset, we highly recommend briefly going through the Perception Synthetic Data Tutorial below!
Human Pose Labeling and Randomization Tutorial
Step by step instructions for using the keypoint, pose, and animation randomization tools included in the Perception package. We recommend that you complete the Perception Tutorial above before diving into this guide!
Analyzing Datasets with Pysolotools
Introduction to Unity's pysolotools - a python package for parsing, converting, and analyzing a SOLO dataset. Step by step instructions on how to parse a SOLO dataset and look at the dataset statistics in a jupyter notebook.
Visualizing a Dataset with Voxel51 Viewer
We will walk through using Voxel51 with our custom SOLO importer to visualize a dataset.
Converting to COCO
We will use pysolotools to convert a SOLO dataset into the COCO format.
Frequently Asked Questions
Check out our FAQ for a list of common questions, tips, tricks, and sample code for common code patterns.
Reference Documentation
In-depth documentation on specific components of the package. For the full set of feature-based documentation, take a look at the Features page.
| Feature | Description |
|---|---|
| Labeling | A component that marks a GameObject and its descendants with a set of labels |
| Label Config | An asset that defines a taxonomy of labels for ground truth generation |
| Perception Camera | Captures RGB images and ground truth from a Camera. |
| Dataset Capture | Ensures sensors are triggered at proper rates and accepts data for the JSON dataset. |
| Accumulation / Path Tracing | Accumulation is a feature that can be used for rendering techniques that require frames to be accumulated, such as path tracing. |
| Randomization | The Randomization tool set lets you integrate domain randomization principles into your simulation. |
| Output Endpoint and Dataset Schema | An endpoint is responsible for delivering the generated data to the user. Currently supported output endpoints are: Solo endpoint (default), Perception endpoint, and No Output. |
Datasets and Sample Projects
Synthetic Homes
Head over to the Synthetic Homes repository for a 100K image dataset of annotated synthetic home interiors for the purpose of training computer vision models, as well as a configurable Unity based stand alone application for generating such datasets.
Synthetic Humans
The Synthetic Humans package gives you the ability to procedurally generate and realistically place diverse groups of synthetic humans in your Unity Computer Vision projects.
People Sans People
PeopleSansPeople is a human-centric privacy-preserving synthetic data generator with highly parametrized domain randomization. This dataset generator contains simulation-ready 3D human assets, a parameterized lighting and camera system, and generates 2D and 3D bounding box, instance and semantic segmentation, and COCO pose labels.
SynthDet
SynthDet is an end-to-end solution for training a 2D object detection model using synthetic data.
Robotics Object Pose Estimation Demo
The Robotics Object Pose Estimation project demonstrates pick-and-place with a robot arm in Unity. It includes using ROS with Unity, importing URDF models, collecting labeled training data using the Perception package, and training and deploying a deep learning model.
Community and Support
Perception is an open-source project and we encourage and welcome contributions. If you wish to contribute, be sure to review our contribution guidelines and code of conduct.
For setup problems or discussions about leveraging the Perception package in your project, please create a new thread on the Unity Computer Vision forums with details such as operating system, reproduction steps, etc. If you run into any other problems with the Perception package or have a specific feature request, please open a GitHub Issue.
For any other questions or feedback, connect directly with the Computer Vision team at [email protected].
License & Citation
The Perception package is licensed under Apache License Version 2.0. If you find this package useful, consider citing it using:
@misc{unity-perception2022,
title={Unity {P}erception Package},
author={{Unity Technologies}},
howpublished={\url{https://github.com/Unity-Technologies/com.unity.perception}},
year={2020}
}