CFL: End-to-End Layout Recovery from 360 Images

Tensorflow implementation of our end-to-end model to recover 3D layouts of indoor scenes.

Introduction

This repository contains original implementation of the paper: 'Corners for Layout: End-to-End Layout Recovery from 360 Images' by Clara Fernandez-Labrador*, José M. Fácil*, Alejandro Perez-Yus, Cédric Demonceaux, Javier Civera and José J. Guerrero.

The page of the paper is https://cfernandezlab.github.io/CFL/

Citing

Please cite CFL in your publications if it helps your research:

@article{fernandez2020corners,
  title={Corners for layout: End-to-end layout recovery from 360 images},
  author={Fernandez-Labrador, Clara and Facil, Jose M and Perez-Yus, Alejandro and Demonceaux, C{\'e}dric and Civera, Javier and Guerrero, Josechu},
  journal={IEEE Robotics and Automation Letters},
  year={2020},
  publisher={IEEE}
}

News

Now implemented for TF2 on the TensorFlow addons package! Check the tutorial in this link!

New implementation in PyTorch as well here link!

Use Instructions

We recommend the use of a virtual enviroment for the use of this project. (e.g. pew)

$ pew new venvname -p python3 # replace venvname with your prefered name (it also works with python 2.7)

Install Requirements

1. This code has been compiled and tested using:

• python3
• g++-4.9
• cuda-10.0
• cuDNN 7.5
• TensorFlow 1.13

You are free to try different configurations. But we do not ensure it had been tested.

2. Install python requirements:

(venvname)$ pip install -r requirements.txt

3. Compile Deformable Convolutions:

We use this Deformable Convolutions implementation to create EquiConvs, a novel implementation of the convolution for 360º images that adapts the shape of the kernel accordingly to the equirectangular projection distortions. Valid for any task involving panoramic images!

First, make sure to modify Models/make.sh with your computer settings (i.e. CUDA_HOME, CUDA Architecture). Second run the following commands.

(venvname)$ cd Models
(venvname)$ bash make.sh

Download Dataset

SUN360: download

Download Models

StdConvs version: download

EquiConvs version: download

Run Test

Edit the config file so that the path to the dataset folder and the network parameters is correct.

To run our demo please run:

(venvname)$ python test_CFL.py --dataset Datasets/SUN360/test --weights Weights/StdConvs/model.ckpt --network StdConvs --results results_std # Runs the test examples with Standard Convolutions and save results in 'results_std'

(venvname)$ python test_CFL.py --dataset Datasets/SUN360/test --weights Weights/EquiConvs/model.ckpt --network EquiConvs --results results_equi # Runs the test examples with Equirectangular Convolutions and save results in 'results_equi'

From Corner Maps to 3D Layout

Check this notebook 2Dto3D.ipynb.

Contact

You can find our contact information in our personal websites Clara and Jose M.

License

This software is under GNU General Public License Version 3 (GPLv3), please see GNU License

For commercial purposes, please contact the authors.

Disclaimer

This site and the code provided here are under active development. Even though we try to only release working high quality code, this version might still contain some issues. Please use it with caution.