luxonis/depthai-unity luxonis/depthai-unity

  • Stars
    star
    200
  • Rank 194,117 (Top 4 %)
  • Language
    C#
  • License
    MIT License
  • Created almost 3 years ago
  • Updated 2 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
luxonis/depthai-unity
github

luxonis

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

DepthAI Unity Library, Unity projects and examples (OAK For Unity)

OAK For Unity

       

License: MIT  unity 2020.3.20f1   unity 2021.2.7f1   unity URP   unity HDRP   unity Builtin  

OAK For Unity is in active development. Its features and API are subject to significant change as development progresses.

Current status (change log):

2023-04-19: Happy to announce Linux support after fixing some big issues regarding Unity editor and binary export.

**Special thanks to @sliwowitz and @onuralpszr for their contribution and patience with Linux support !

  • Added pre-compiled library (Ubuntu 20.04.5 LTS)
  • Added instructions how to compile library

2023-04-18:

  • Bump depthai-core to 2.21.2
  • Prepare for Linux support
  • Update face emotion models to supported OpenVINO version

2023-01-19: Support for active vision - oak-d-pro

2022-11-11: Face emotion demo scene for WINDOWS.

2022-11-07: Support for OAK-PRO and OAK-POE devices.

2022-06-05: Body pose demo scene for WINDOWS.

Expand to read more ...

2022-05-04: Fix point cloud issues when export as binary. Color point cloud.

2022-03-10: Object detector demo scene for WINDOWS.

2022-02-21: Face detector demo scene for WINDOWS.

2022-01-19: Demo menu scene added. Now it's much easier to navigate throught the demos. Go to menu on top: "OAK For Unity"->"Example scenes" and hit play

2022-01-14: Device manager, basic streams and point cloud for WINDOWS. Working on rest platforms (MacOS/Linux)


Now it's much easier to navigate throught the demos. Go to menu on top: "OAK For Unity"->"Example scenes" and hit play

detailmenu

What is OAK For Unity? (Click here for more detail)

OAK For Unity is native plugin for Windows, Linux and MacOS (Android support ongoing) to enable OAK devices and capabilities inside Unity. Main goal is bring the power of OAK devices, CV and Edge AI to Unity community to build the next generation of Spatial AI / Edge AI applications.

Expand to read more ...

OAK For Unity Creators: Build interactive videogames and experiences with pretrained models and high-level API

OAK For Unity Developers: Productivity tools for Unity developers: VR workflows, MoCap tools (body pose and face mesh)

OAK For CV/AI/Robotics Developers: Virtual sensors, simulation, custom pipelines, ROS integration, synthetic dataset generation with automatic labelling, Deep RL, ...

This repository contains everything you need to integrate OAK with Unity.

  • Unity projects and examples
  • depthai-unity C/C++ library (source code and precompiled binaries)
  • .unitypackage to easy import OAK For Unity into your project (Coming soon)
  • Notice there is no C# wrapper available at this moment. We plan to do partial C# wrapper of depthai-core library in the future.

Help build the roadmap checking out our Roadmap discussion and feel free to explain about your use case.

OAK For Creators. Unity plugin and pretrained models.

Getting started

Quick Installation Instructions

Get your local workspace up and running quickly (less than 30 secs)

OAK For Unity

Everything related to OAK For Unity package, projects and demo scenes.

This repo contains precompiled versions of depthai-unity library lowering any entry barrier, so you don't need to deal with C/C++ compilation.

Export Unity binary

Notice when you export binary, Unity is not doing copy of models (URP/Assets/Plugin/OAKForUnity/Models)

If you export example or scene using neural model like face detector, face emotion or body pose, you need to copy models folder content to URP_Data/Plugins/OAKForUnity/Models

Prebuild Unity demos

Do you want to try some of our demos?

Do you need to update depthai-unity? Do you want to integrate your own pipeline using depthai-unity library? We got you covered on the following section.

Building depthai-unity (OPTIONAL)

Building depthai-unity library

Detailed instructions covering all steps to build depthai-unity C/C++ library (based on depthai-core C++ library) for all the platforms

Tutorial: Integrating your own pipeline inside Unity - Coming soon

Step-by-step tutorial how to integrate your own pipeline inside Unity. Please check how to build depthai-unity library first if you didn't.

FAQ - Coming soon

Check out our FAQ for a list of common questions, tips, tricks, and some sample code.

Documentation

In-depth documentation on individual components of the package

Community and Support

If you run into any other problems with the OAK For Unity or have a specific feature request, please submit a GitHub issue.

Example projects

  • HDRP project (Coming soon)
  • URP project (Coming soon)
  • Integration with Unity Perception package for synthetic dataset with automatic labelling (Coming soon)
  • Integration with Unity Robotics Hub package and ROS (Coming soon)

Community Projects

Are you building spatial app using OAK For Unity? Please DM and will be a pleasure to add a reference here

Compatibility

Platform Unity Render Pipeline
Windows 2021.2.7f1 ALL
MacOS 2021.2.7f1 ALL
Linux 2021.3.22f1 ALL (tested URP)

Known issues

  • If you're using OAK-1 (don't have stereo depth support) you need to disable depth on the examples, to prevent crash. UseDepth = false; config.confidenceThreshold = 0;

  • If you just use the precompiled depthai-unity library inside Unity, be sure you're using latest version.

Related links

License

OAK For Unity is licensed under MIT License. See LICENSE for the full license text.

More Repositories

1

depthai

DepthAI Python API utilities, examples, and tutorials.
Python
901
star
2

depthai-experiments

Experimental projects we've done with DepthAI.
Jupyter Notebook
800
star
3

depthai-hardware

Altium Designs for DepthAI Carrier Boards
HTML
433
star
4

depthai-python

DepthAI Python Library
C++
340
star
5

depthai-ros

Official ROS Driver for DepthAI Sensors.
C++
239
star
6

depthai-core

DepthAI C++ Library
C++
227
star
7

depthai-ml-training

Some Example Neural Models that we've trained along with the training scripts
Jupyter Notebook
118
star
8

datadreamer

Creation of annotated datasets from scratch using Generative AI and Foundation Computer Vision models
Python
76
star
9

depthai-gui

DepthAI Pipeline Builder GUI
Python
71
star
10

depthai-ros-examples

Python
40
star
11

depthai-tutorials

Source code for DepthAI tutorials published @ https://docs.luxonis.com
Python
39
star
12

depthai-model-zoo

DepthAI Model Zoo is a collection of open-source neural network models and datasets created and maintained by DepthAI developers and community
SCSS
34
star
13

esp32-spi-message-demo

ESP32 reference app for interfacing with DepthAI over SPI
C++
31
star
14

rae-ros

Implementation of RAE ROS and gazebo stack
Python
30
star
15

tools

Various tools for OAK-D camera
Jupyter Notebook
26
star
16

depthai-docs-website

The documentation site for Luxonis DepthAI
CSS
25
star
17

Factory-calibration-DepthAI

Factory Calibration for DepthAI Stereo-capable models.
Python
17
star
18

depthai-shared

DepthAI Shared code and data
C++
16
star
19

blobconverter

Web-based tool to convert model into MyriadX blob
Python
15
star
20

depthai-core-example

CMake C++ example project using depthai library
CMake
14
star
21

models

Repository of lightweight models for classification, object detection, segmentation and more. Deployable to OAK-D, OpenVINO, and ONNX.
Python
14
star
22

luxonis-ml

Luxonis ML library which abstracts logging, tracking, and other useful functionalities.
Python
13
star
23

DepthAI-PyFlow

OUTDATED, see here - https://github.com/luxonis/depthai-gui
Python
11
star
24

XLink

A cross-platform library for communicating with devices over various physical links.
C
11
star
25

yolo2openvino

YoloV3, YoloV4, YoloV3-tiny, and YoloV4-tiny conversion to Tensorflow and OpenVINO.
Python
10
star
26

depthai-docker

Python
8
star
27

luxonis-train

Training framework for easy creation, training and exporting of deep learning models.
Python
8
star
28

remote-monitoring

JavaScript
7
star
29

depthai-poe-webapp

DepthAI WebApp for PoE devices
Python
7
star
30

robothub-images

Shell
6
star
31

modelconverter

Model converter for Luxonis' cameras. Convert your model from ONNX, TF, ... to a model compatible with any generation of Luxonis camera.
Python
6
star
32

depthai-spi-api

C++
4
star
33

robothub-examples

Python
3
star
34

robothub-public-apps

Python
3
star
35

depthai-boards

Files with hardware descriptions for depthai boards
Python
3
star
36

simulation

3
star
37

depthai-calibration

Python
2
star
38

depthai-ros-release

Release repository of depthai-ros
2
star
39

depthai-spi-library

DepthAI SPI Library
C
2
star
40

depthai-bootloader-shared

Shared code and data for depthai-bootloader
C++
2
star
41

python-api-analyzer-to-json

Python
1
star
42

robothub

Python
1
star
43

SBR

Library and utility to view and manipulate SBR images
C
1
star
44

tusb926x-flash-burner

TUSB926x modern libusb/hidapi Linux flasher
C++
1
star
45

depthai-node

CMake
1
star
46

rp2040_u2if

Interface with RP2040 running U2IF FW
Python
1
star
47

rae-unity

RAE Unity plugin and Simulation
1
star