3D reconstruction

3D reconstruction from 2D images pipeline

Steps:

1. Detect 2D points
2. Match 2D points across 2 images
3. Epipolar geometry
   3a. If both intrinsic and extrinsic camera parameters are known, reconstruct with projection matrices.
   3b. If only the intrinsic parameters are known, normalize coordinates and calculate the essential matrix.
   3c. If neither intrinsic nor extrinsic parameters are known, calculate the fundamental matrix.
4. With fundamental or essential matrix, assume P1 = [I 0] and calulate parameters of camera 2.
5. Triangulate knowing that x1 = P1 * X and x2 = P2 * X.
6. Bundle adjustment to minimize reprojection errors and refine the 3D coordinates.

Note: Steps and code in this repo is my hobby / learning exercise. Ie, its probably not very efficient. If you wish to use a more production-ready library, check out OpenCV's SFM module. I have a docker environment for it at: https://github.com/alyssaq/reconstruction

Prerequisites

• Python 3.5+
• Install OpenCV: Mac installation steps
• pip install -r requirements.txt

Example 3D cube reconstruction

$ python3 cube_reconstruction.py

Example Dino 3D reconstruction from 2D images

Download images from https://www.robots.ox.ac.uk/~vgg/data/mview/ and place into imgs/dinos

$ python3 example.py

Detected points and matched across 2 images.

3D reconstructed dino with essential matrix

3D to 2D Projection

$ python3 camera.py

3D points of model house from Oxford University VGG datasets.

Projected points

Datasets

• Oxford University, Visual Geometry Group: http://www.robots.ox.ac.uk/~vgg/data/data-mview.html
• EPFL computer vision lab: http://cvlabwww.epfl.ch/data/multiview/knownInternalsMVS.html

References

• Eight point algorithm
• Multiple View Geometry in Computer Vision (Hartley & Zisserman)

License

MIT