OAK For Unity
Β Β
Β Β
OAK For Unityis in active development. Its features and API are subject to significant change as development progresses.
Current status (change log):
2023-04-19: Happy to announce Linux support after fixing some big issues regarding Unity editor and binary export.
**Special thanks to @sliwowitz and @onuralpszr for their contribution and patience with Linux support !
- Added pre-compiled library (Ubuntu 20.04.5 LTS)
- Added instructions how to compile library
2023-04-18:
- Bump depthai-core to 2.21.2
- Prepare for Linux support
- Update face emotion models to supported OpenVINO version
2023-01-19: Support for active vision - oak-d-pro
2022-11-11: Face emotion demo scene for WINDOWS.
2022-11-07: Support for OAK-PRO and OAK-POE devices.
2022-06-05: Body pose demo scene for WINDOWS.
Expand to read more ...
2022-05-04: Fix point cloud issues when export as binary. Color point cloud.
2022-03-10: Object detector demo scene for WINDOWS.
2022-02-21: Face detector demo scene for WINDOWS.
2022-01-19: Demo menu scene added. Now it's much easier to navigate throught the demos. Go to menu on top: "OAK For Unity"->"Example scenes" and hit play
2022-01-14: Device manager, basic streams and point cloud for WINDOWS. Working on rest platforms (MacOS/Linux)
Now it's much easier to navigate throught the demos. Go to menu on top: "OAK For Unity"->"Example scenes" and hit play
What is OAK For Unity? (Click here for more detail)
OAK For Unity is native plugin for Windows, Linux and MacOS (Android support ongoing) to enable OAK devices and capabilities inside Unity. Main goal is bring the power of OAK devices, CV and Edge AI to Unity community to build the next generation of Spatial AI / Edge AI applications.
Expand to read more ...
OAK For Unity Creators: Build interactive videogames and experiences with pretrained models and high-level API
OAK For Unity Developers: Productivity tools for Unity developers: VR workflows, MoCap tools (body pose and face mesh)
OAK For CV/AI/Robotics Developers: Virtual sensors, simulation, custom pipelines, ROS integration, synthetic dataset generation with automatic labelling, Deep RL, ...
This repository contains everything you need to integrate OAK with Unity.
- Unity projects and examples
- depthai-unity C/C++ library (source code and precompiled binaries)
- .unitypackage to easy import OAK For Unity into your project (Coming soon)
- Notice there is no C# wrapper available at this moment. We plan to do partial C# wrapper of depthai-core library in the future.
Help build the roadmap checking out our Roadmap discussion and feel free to explain about your use case.
OAK For Creators. Unity plugin and pretrained models.
Getting started
Quick Installation Instructions
Get your local workspace up and running quickly (less than 30 secs)
Everything related to OAK For Unity package, projects and demo scenes.
This repo contains precompiled versions of depthai-unity library lowering any entry barrier, so you don't need to deal with C/C++ compilation.
Export Unity binary
Notice when you export binary, Unity is not doing copy of models (URP/Assets/Plugin/OAKForUnity/Models)
If you export example or scene using neural model like face detector, face emotion or body pose, you need to copy models folder content to URP_Data/Plugins/OAKForUnity/Models
Do you want to try some of our demos?
Do you need to update depthai-unity? Do you want to integrate your own pipeline using depthai-unity library? We got you covered on the following section.
Building depthai-unity (OPTIONAL)
Building depthai-unity library
Detailed instructions covering all steps to build depthai-unity C/C++ library (based on depthai-core C++ library) for all the platforms
Tutorial: Integrating your own pipeline inside Unity - Coming soon
Step-by-step tutorial how to integrate your own pipeline inside Unity. Please check how to build depthai-unity library first if you didn't.
Check out our FAQ for a list of common questions, tips, tricks, and some sample code.
Documentation
In-depth documentation on individual components of the package
Community and Support
If you run into any other problems with the OAK For Unity or have a specific feature request, please submit a GitHub issue.
Example projects
- HDRP project (Coming soon)
- URP project (Coming soon)
- Integration with Unity Perception package for synthetic dataset with automatic labelling (Coming soon)
- Integration with Unity Robotics Hub package and ROS (Coming soon)
Community Projects
Are you building spatial app using OAK For Unity? Please DM and will be a pleasure to add a reference here
- jbb-kryo is building Unity app with some support for HoloLens2 and MKRT. Take a look here: https://github.com/kryotech-ltd/depthai-unity/tree/mkrt-hl2-update
Compatibility
| Platform | Unity | Render Pipeline |
|---|---|---|
| Windows | 2021.2.7f1 | ALL |
| MacOS | 2021.2.7f1 | ALL |
| Linux | 2021.3.22f1 | ALL (tested URP) |
Known issues
-
If you're using OAK-1 (don't have stereo depth support) you need to disable depth on the examples, to prevent crash. UseDepth = false; config.confidenceThreshold = 0;
-
If you just use the precompiled depthai-unity library inside Unity, be sure you're using latest version.
Related links
License
OAK For Unity is licensed under MIT License. See LICENSE for the full license text.