PoC - Master Thesis 
Optimierung von Augmented Reality Anwendungen durch die Berücksichtigung von Tiefeninformationen mit Googles Project Tango
Optimization of AR applications considering the depth information with Googles project tango
Table of Content
- Final Prototoype
- Augmented Reality Demo
- Pointcloud Extraction
- Plane based Reconstruction
- PCL based native Reconstruction (Greedy Projection Triangulation)
- CHISEL based native reconstruction
- Unity Marching Cubes AR Demo
- OpenCV Depthmap Generation and Filtering
- OpenCV Depthmap Filtering for AR
Final Prototype
- depthmap rendering with 3 different reconstruction mechanisms
- pointcloud projection
- tsdf reconstruction
- plane reconstruction
- using depthmap as Z-Buffer (
GL_DEPTH_COMPONENT) for augmented reality occlustion
- optional OpenCV
guided filteringwith settings possible
Augmented Reality (ar/)
- AR Camera matching intrinsics
- motion tracking with correct gl positioning
- visualization of the current pointcloud
- ray-intersection for pointcloud interaction
- simple tower defense game with ray intersection
- simple direct pointcloud z-buffer occlusion
Pointcloud App (pc/)
- collects points into a large pointcloud using an octree
- exports to pts file
- plane based reconstruction applicable
- marching cube reconstruction applicable (usualy to slow here!)
Plane Based Reconstruction (construct/)
1. OctTree clustering of the global pointcloud
2. on incoming depth frame
* update random n global clusters with points from depth frame
* use RANSAC to detect 3 planes per cluster
* project points from 3D space to 2D space (based on detected planes)
* use Graham Scan to compute comvex hull for each plane
* use Sweep‐line for triangulation with poly2tri
* project polygon vertices back to 3D space (based on detected planes)
Video Demonstration
Old Screenshots
Native PCL based Reconstruction (construct-native/)
- VoxelGrid downsampling
- Greedy Triangulation with PCL
- Build PCL Library not included!
CHISEL open_chisel implementation (chisel/)
- online reconstruction using
open_chisellibrary - transformation and interfaces to native environment
- PLY exporter
- Build open_chisel Library not included!
OpenCV Depthmap Generation and Filtering
- generate depthmap using this approach
- applying inpaint and guided filtering for depthmap improvements
- Build opencv+contrib Library not included!
Video Demonstration
OpenCV Depthmap Filtering for AR
- using the previous approach for a pointcloud reprojection
Unity Implementierung (unity/)
- Reconstruction using the MeshBuilder with an simple TSDF by Google
- Implementation of a Clipping Depth Shader
- simple ball moving interaction
- extension of shader for shadowing effects
Video Demonstration
Old Screenshots
