ZED Unity Plugin

This package brings the features of the ZED cameras into Unity. It comes with prefabs to make adding advanced features to your project as simple as drag-and-drop, helper scripts to simplify custom integration, and numerous examples to see your camera in action and learn how it works.

Overview

Body Tracking	AR/MR	Point Cloud
Object Detection	Dark Room	Spatial mapping
Object Placement	ArUco Markers	Green Screen

Compatibility

ZED Camera	GPU	AR/VR
Any ZED camera	CUDA 10.2 capability or higher GTX 1060 or higher is recommended	HMDs compatible with OpenVR or Oculus
OS	Unity Version	ZED SDK Version
Windows 10/11 Ubuntu 18-22	Unity 2021.3 and newer	ZED SDK 4.0 Previous versions are supported by previous plugins

Get Started

To develop applications in Unity with your ZED, you'll need the following things:

• Download and install the ZED SDK.
• Download the ZED Plugin for Unity.
• Import the Zed_Unity_Plugin_V4.0.X.unitypackage into Unity.
  • If you get an error saying The type or namespace name ‘Management’ does not exist in the namespace ‘UnityEngine.XR’, please go to Project Settings -> XR Plugin Management and Install the XR Plugin Management system by clicking the button.
• Read the Basic Concepts and Build Your First AR/MR App guides to get started.
• Explore the Samples included in the plugin.
• The Main Scripts page of the documentation introduces the important scripts of the plugin and their parameters.

Example Scenes

After importing the plugin, pick and try some of the example scenes. Each is designed to demonstrate a main feature of the ZED. Some contain prefabs and example scripts that can be repurposed for your projects.

The overview is available in the documentation for the ZED Unity plugin.

• Body Tracking: Animate a 3D avatar based on real-people movements using the ZED SDK Body Tracking module.
• Dark Room: Your office is now a night club! Explore the lighting features coupled to the depth sensing to cast a laser show on your walls.
• Drone Shooter: Defend yourself from drones that spawn around your room and shoot at you. Block lasers with your hand, and shoot back with the spacebar, or VR controllers if using the Oculus Integration or SteamVR plugin.
• Green Screen: Aim your ZED at a greenscreen and hit Play to see your subject standing in a small town in the desert. You’ll see that the nearby crates still have all the proper occlusion, but the greenscreen background is replaced with the virtual background.
• Movie Screen: Simple AR sample, playing a movie on a movable, scalable 3D screen integrated to the real-world. Demonstrates how 2D content can easily be displayed in a 3D, mixed reality scene.
• Multicam: Integrate data from multiple ZED cameras in the same scene, at the same time.
• Object Detection: Detect objects and people bounding boxes using the ZED SDK Object Detection module.
• Object Placement: Place virtual objects on real-world surfaces using the Plane Detection capabilities of the ZED SDK.
• OpenCV ArUco Detection: Print out ArUco markers, put them in view of your ZED, and let the battle of the drones begin. Shows how to easily interface the ZED with OpenCV for marker detection using a variety of included scripts.
• Plane Detection: Run the scene and hold down spacebar to see if you’re looking at a valid surface where a bunny could stand. Release the spacebar and a bunny will fall from the sky and land on that surface with proper physics.
• VR Plane Detection: Throw a plushie as far as possible, the distance is measured where it lands using the Plane Detection. Shows how plane detection can fit into a proper VR game.
• Planetarium: A beautiful display of the ZED plugin’s basic mixed reality features, viewable with or without a headset. Watch as the planets are properly occluded by the real world.
• Point Cloud: Visualize the depth retrieved by the ZED Camera directly into Unity, in the form of a Point Cloud.
• Simple MR: Learn to create a basic AR app, with depth occlusion and camera tracking.
• Spatial Mapping: Capture a mesh of your environment, allowing physical interactions between the real and virtual world.

Additional Resources

Want more details? Our 📖 Documentation has overviews and tutorials for getting started with specific features.

Got a problem you just can't seem to solve? Got a project you want to share? Check out our Community forums!

Bugs and fixes

You found a bug / a flaw in our plugin ? Please check that it is not already reported, and open an issue if necessary. You can also reach out to us on the community forums for any question or feedback !

By the way, we also have a special place in our hearts for PR senders.