Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges

This is the official repository of the SensatUrban dataset. For technical details, please refer to:

Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
Qingyong Hu, Bo Yang*, Sheikh Khalid, Wen Xiao, Niki Trigoni, Andrew Markham.
[Paper] [Blog] [Video] [Project page] [Download] [Evaluation] [Urban3D workshop@ECCV2022]

We are hosting the 2nd Urban3D Challenge at ECCV 2022, please refer to this page for more details.

(1) Dataset

1.1 Overview

This dataset is an urban-scale photogrammetric point cloud dataset with nearly three billion richly annotated points, which is five times the number of labeled points than the existing largest point cloud dataset. Our dataset consists of large areas from two UK cities, covering about 6 km^2 of the city landscape. In the dataset, each 3D point is labeled as one of 13 semantic classes, such as ground, vegetation, car, etc..

1.2 Data Collection

The 3D point clouds are generated from high-quality aerial images captured by a professional-grade UAV mapping system. In order to fully and evenly cover the survey area, all flight paths are pre-planned in a grid fashion and automated by the flight control system (e-Motion).

1.3 Semantic Annotations

Ground: including impervious surfaces, grass, terrain
Vegetation: including trees, shrubs, hedges, bushes
Building: including commercial / residential buildings
Wall: including fence, highway barriers, walls
Bridge: road bridges
Parking: parking lots
Rail: railroad tracks
Traffic Road: including main streets, highways
Street Furniture: including benches, poles, lights
Car: including cars, trucks, HGVs
Footpath: including walkway, alley
Bike: bikes / bicyclists
Water: rivers / water canals

1.4 Statistics

(2) Benchmarks

We extensively evaluate the performance of state-of-the-art algorithms on our dataset and provide a comprehensive analysis of the results. In particular, we identify several key challenges towards urban-scale point cloud understanding.

(3) Demo

(4) Training and Evaluation

Here we provide the training and evaluation script of RandLA-Net for your reference.

Download the dataset

Download the files named "data_release.zip" here. Uncompress the folder and move it to /Dataset/SensatUrban.

Setup the environment

conda create -n randlanet python=3.5
source activate randlanet
pip install -r helper_requirements.txt
sh compile_op.sh

Preparing the dataset

python input_preparation.py --dataset_path $YOURPATH
cd $YOURPATH; 
cd ../; mkdir original_block_ply; mv data_release/train/* original_block_ply; mv data_release/test/* original_block_ply;
mv data_release/grid* ./

The data should organized in the following format:

/Dataset/SensatUrban/
          └── original_block_ply/
                  ├── birmingham_block_0.ply
                  ├── birmingham_block_1.ply 
		  ...
	    	  └── cambridge_block_34.ply 
          └── grid_0.200/
	     	  ├── birmingham_block_0_KDTree.pkl
                  ├── birmingham_block_0.ply
		  ├── birmingham_block_0_proj.pkl 
		  ...
	    	  └── cambridge_block_34.ply

Start training: (Please first modified the root_path)

python main_SensatUrban.py --mode train --gpu 0

Evaluation:

python main_SensatUrban.py --mode test --gpu 0

Submit the results to the server: The compressed results can be found in /test/Log_*/test_preds/submission.zip. Then, feel free to submit this results to the evaluation server.
The Urban3D Challenge@ICCV2021 Forum: Please scan the code to join our wechat group or drop a message here:

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{hu2020towards,
  title={Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges},
  author={Hu, Qingyong and Yang, Bo and Khalid, Sheikh and Xiao, Wen and Trigoni, Niki and Markham, Andrew},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2021}
}


@article{hu2022sensaturban,
title={Sensaturban: Learning semantics from urban-scale photogrammetric point clouds},
author={Hu, Qingyong and Yang, Bo and Khalid, Sheikh and Xiao, Wen and Trigoni, Niki and Markham, Andrew},
journal={International Journal of Computer Vision},
volume={130},
number={2},
pages={316--343},
year={2022},
publisher={Springer}

}

Updates

03/25/2022: we are organizing the Urban3D@ECCV2022 - The 2nd Challenge on Large-Scale Point Clouds Analysis for Urban Scenes Understanding!
01/03/2021: The SensatUrban has been accepted by CVPR 2021!
11/02/2021: The dataset is available for download!
07/09/2020: Initial release!