Deep Learning for 3D Point Cloud Understanding: A Survey
Our survey paper[ArXiv]
@article{lu2020deep,
title={Deep Learning for 3D Point Cloud Understanding: A Survey},
author={Lu, Haoming and Shi, Humphrey},
journal={arXiv preprint arXiv:2009.08920},
year={2020}
}
Content
- Datasets
- Metrics
- Papers
Datasets
Metrics
Papers (up to ECCV 2020)
3D Object Classification
Projection-based classification
-
Su, Hang, et al. "Multi-view convolutional neural networks for 3d shape recognition." Proceedings of the IEEE international conference on computer vision. 2015. [paper]
-
Feng, Yifan, et al. "Gvcnn: Group-view convolutional neural networks for 3d shape recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Yu, Tan, Jingjing Meng, and Junsong Yuan. "Multi-view harmonized bilinear network for 3d object recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Yang, Ze, and Liwei Wang. "Learning relationships for multi-view 3D object recognition." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Maturana, Daniel, and Sebastian Scherer. "Voxnet: A 3d convolutional neural network for real-time object recognition." 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2015. [paper]
-
Wu, Zhirong, et al. "3d shapenets: A deep representation for volumetric shapes." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. [paper]
-
Riegler, Gernot, Ali Osman Ulusoy, and Andreas Geiger. "Octnet: Learning deep 3d representations at high resolutions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017. [paper]
-
Prokudin, Sergey, Christoph Lassner, and Javier Romero. "Efficient learning on point clouds with basis point sets." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
Point-based classification
-
Qi, Charles R., et al. "Pointnet: Deep learning on point sets for 3d classification and segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. [paper]
-
Qi, Charles Ruizhongtai, et al. "Pointnet++: Deep hierarchical feature learning on point sets in a metric space." Advances in neural information processing systems. 2017. [paper]
-
Zhao, Hengshuang, et al. "PointWeb: Enhancing local neighborhood features for point cloud processing." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Duan, Yueqi, et al. "Structural relational reasoning of point clouds." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Komarichev, Artem, Zichun Zhong, and Jing Hua. "A-CNN: Annularly convolutional neural networks on point clouds." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Liu, Yongcheng, et al. "Relation-shape convolutional neural network for point cloud analysis." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Wu, Wenxuan, Zhongang Qi, and Li Fuxin. "Pointconv: Deep convolutional networks on 3d point clouds." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Hermosilla, Pedro, et al. "Monte carlo convolution for learning on non-uniformly sampled point clouds." ACM Transactions on Graphics (TOG) 37.6 (2018): 1-12. [paper]
-
Lan, Shiyi, et al. "Modeling local geometric structure of 3D point clouds using Geo-CNN." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Rao, Yongming, Jiwen Lu, and Jie Zhou. "Spherical fractal convolutional neural networks for point cloud recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Simonovsky, Martin, and Nikos Komodakis. "Dynamic edge-conditioned filters in convolutional neural networks on graphs." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. [paper]
-
Wang, Yue, et al. "Dynamic graph cnn for learning on point clouds." Acm Transactions On Graphics (tog) 38.5 (2019): 1-12. [paper]
-
Hassani, Kaveh, and Mike Haley. "Unsupervised multi-task feature learning on point clouds." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Chen, Chao, et al. "Clusternet: Deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Klokov, Roman, and Victor Lempitsky. "Escape from cells: Deep kd-networks for the recognition of 3d point cloud models." Proceedings of the IEEE International Conference on Computer Vision. 2017. [paper]
-
Zeng, Wei, and Theo Gevers. "3DContextNet: Kd tree guided hierarchical learning of point clouds using local and global contextual cues." Proceedings of the European Conference on Computer Vision (ECCV). 2018. [paper]
-
Wu, Pengxiang, et al. "Point cloud processing via recurrent set encoding." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. 2019. [paper]
-
Li, Jiaxin, Ben M. Chen, and Gim Hee Lee. "So-net: Self-organizing network for point cloud analysis." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. [paper]
3D Segmentation
Semantic segmentation
-
Huang, Jing, and Suya You. "Point cloud labeling using 3d convolutional neural network." 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, 2016. [paper]
-
Dai, Angela, et al. "Scancomplete: Large-scale scene completion and semantic segmentation for 3d scans." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Meng, Hsien-Yu, et al. "Vv-net: Voxel vae net with group convolutions for point cloud segmentation." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Lawin, Felix Järemo, et al. "Deep projective 3D semantic segmentation." International Conference on Computer Analysis of Images and Patterns. Springer, Cham, 2017. [paper]
-
Zhang, Yang, et al. "PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
-
Dai, Angela, and Matthias Nießner. "3dmv: Joint 3d-multi-view prediction for 3d semantic scene segmentation." Proceedings of the European Conference on Computer Vision (ECCV). 2018. [paper]
-
Jaritz, Maximilian, Jiayuan Gu, and Hao Su. "Multi-view pointnet for 3d scene understanding." Proceedings of the IEEE International Conference on Computer Vision Workshops. 2019. [paper]
-
Engelmann, Francis, et al. "Know what your neighbors do: 3D semantic segmentation of point clouds." Proceedings of the European Conference on Computer Vision (ECCV). 2018. [paper]
-
Wang, Shenlong, et al. "Deep parametric continuous convolutional neural networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Liu, Zhijian, et al. "Point-Voxel CNN for efficient 3D deep learning." Advances in Neural Information Processing Systems. 2019. [paper]
-
Hua, Binh-Son, Minh-Khoi Tran, and Sai-Kit Yeung. "Pointwise convolutional neural networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Landrieu, Loic, and Martin Simonovsky. "Large-scale point cloud semantic segmentation with superpoint graphs." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Landrieu, Loic, and Mohamed Boussaha. "Point cloud oversegmentation with graph-structured deep metric learning." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Wang, Lei, et al. "Graph attention convolution for point cloud semantic segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Tatarchenko, Maxim, et al. "Tangent convolutions for dense prediction in 3d." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Hu, Qingyong, et al. "RandLA-Net: Efficient semantic segmentation of large-scale point clouds." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
-
Xu, Xun, and Gim Hee Lee. "Weakly Supervised Semantic Point Cloud Segmentation: Towards 10x Fewer Labels." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
-
Wei, Jiacheng, et al. "Multi-Path Region Mining For Weakly Supervised 3D Semantic Segmentation on Point Clouds." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
Instance segmentation
-
Hou, Ji, Angela Dai, and Matthias Nießner. "3d-sis: 3d semantic instance segmentation of rgb-d scans." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Yi, Li, et al. "Gspn: Generative shape proposal network for 3d instance segmentation in point cloud." Proceedings of the IEEE conference on computer vision and pattern recognition. 2019. [paper]
-
Yang, Ze, and Liwei Wang. "Learning relationships for multi-view 3D object recognition." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Zhang, Feihu, et al. "Instance segmentation of lidar point clouds." ICRA, Cited by 4.1 (2020). [paper]
-
Wang, Weiyue, et al. "Sgpn: Similarity group proposal network for 3d point cloud instance segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Lahoud, Jean, et al. "3d instance segmentation via multi-task metric learning." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Zhang, Biao, and Peter Wonka. "Point cloud instance segmentation using probabilistic embeddings." arXiv preprint arXiv:1912.00145 (2019). [paper]
-
Wu, Bichen, et al. "Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud." 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2018. [paper]
-
Wu, Bichen, et al. "Squeezesegv2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a lidar point cloud." 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019. [paper]
-
Lyu, Yecheng, Xinming Huang, and Ziming Zhang. "Learning to Segment 3D Point Clouds in 2D Image Space." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
-
Jiang, Li, et al. "PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
Joint training
- Hassani, Kaveh, and Mike Haley. "Unsupervised multi-task feature learning on point clouds." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
- Pham, Quang-Hieu, et al. "JSIS3D: joint semantic-instance segmentation of 3d point clouds with multi-task pointwise networks and multi-value conditional random fields." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
- Wang, Xinlong, et al. "Associatively segmenting instances and semantics in point clouds." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
3D Object Detection
Projection-based detection
- Zhou, Yin, and Oncel Tuzel. "Voxelnet: End-to-end learning for point cloud based 3d object detection." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
- Yan, Yan, Yuxing Mao, and Bo Li. "Second: Sparsely embedded convolutional detection." Sensors 18.10 (2018): 3337. [paper]
- Lang, Alex H., et al. "Pointpillars: Fast encoders for object detection from point clouds." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
- Wang, Yue, et al. "Pillar-based Object Detection for Autonomous Driving." Proceedings of the European Conference on Computer Vision (ECCV). (2020). [paper]
- He, Chenhang, et al. "Structure Aware Single-stage 3D Object Detection from Point Cloud." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
Point-based detection
- Yang, Zetong, et al. "Std: Sparse-to-dense 3d object detector for point cloud." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
- Shi, Shaoshuai, Xiaogang Wang, and Hongsheng Li. "Pointrcnn: 3d object proposal generation and detection from point cloud." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
- Qi, Charles R., et al. "Deep hough voting for 3d object detection in point clouds." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
- Qi, Charles R., et al. "Imvotenet: Boosting 3d object detection in point clouds with image votes." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
- Du, Liang, et al. "Associate-3Ddet: Perceptual-to-Conceptual Association for 3D Point Cloud Object Detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
- Yang, Zetong, et al. "3dssd: Point-based 3d single stage object detector." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
- Zarzar, Jesus, Silvio Giancola, and Bernard Ghanem. "PointRGCN: Graph convolution networks for 3D vehicles detection refinement." arXiv preprint arXiv:1911.12236 (2019). [paper]
- Chen, Jintai, et al. "A Hierarchical Graph Network for 3D Object Detection on Point Clouds." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
- Shi, Weijing, and Raj Rajkumar. "Point-gnn: Graph neural network for 3d object detection in a point cloud." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
Multi-view fusion
-
Chen, Xiaozhi, et al. "Multi-view 3d object detection network for autonomous driving." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017. [paper]
-
Liang, Ming, et al. "Deep continuous fusion for multi-sensor 3d object detection." Proceedings of the European Conference on Computer Vision (ECCV). 2018. [paper]
-
Lu, Haihua, et al. "SCANet: Spatial-channel attention network for 3D object detection." ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019. [paper]
-
Zeng, Yiming, et al. "Rt3d: Real-time 3-d vehicle detection in lidar point cloud for autonomous driving." IEEE Robotics and Automation Letters 3.4 (2018): 3434-3440. [paper]
-
Qi, Charles R., et al. "Frustum pointnets for 3d object detection from rgb-d data." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. [paper]
-
Gupta, Ayush. "Deep Sensor Fusion for 3D Bounding Box Estimation and Recognition of Objects." [paper]
3D Object Tracking
- Giancola, Silvio, Jesus Zarzar, and Bernard Ghanem. "Leveraging shape completion for 3d siamese tracking." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
- Zarzar, Jesus, Silvio Giancola, and Bernard Ghanem. "PointRGCN: Graph convolution networks for 3D vehicles detection refinement." arXiv preprint arXiv:1911.12236 (2019). [paper]
- Chiu, Hsu-kuang, et al. "Probabilistic 3d multi-object tracking for autonomous driving." arXiv preprint arXiv:2001.05673 (2020). [paper]
- Qi, Haozhe, et al. "P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
3D Scene Flow Estimation
- Liu, Xingyu, Charles R. Qi, and Leonidas J. Guibas. "Flownet3d: Learning scene flow in 3d point clouds." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
- Wang, Zirui, et al. "FlowNet3D++: Geometric losses for deep scene flow estimation." The IEEE Winter Conference on Applications of Computer Vision. 2020. [paper]
- Gu, Xiuye, et al. "Hplflownet: Hierarchical permutohedral lattice flownet for scene flow estimation on large-scale point clouds." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
- Liu, Xingyu, Mengyuan Yan, and Jeannette Bohg. "Meteornet: Deep learning on dynamic 3d point cloud sequences." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
- Fan, Hehe, and Yi Yang. "PointRNN: Point recurrent neural network for moving point cloud processing." arXiv preprint arXiv:1910.08287 (2019). [paper]
3D Point Registration and Matching
-
Lu, Weixin, et al. "Deepvcp: An end-to-end deep neural network for point cloud registration." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Gojcic, Zan, et al. "The perfect match: 3d point cloud matching with smoothed densities." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Gojcic, Zan, et al. "Learning multiview 3D point cloud registration." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
-
Yew, Zi Jian, and Gim Hee Lee. "RPM-Net: Robust Point Matching using Learned Features." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
Point Cloud Augmentation and Completion
Discriminative methods
-
Rakotosaona, Marie‐Julie, et al. "Pointcleannet: Learning to denoise and remove outliers from dense point clouds." Computer Graphics Forum. Vol. 39. No. 1. 2020. [paper]
-
Guerrero, Paul, et al. "Pcpnet learning local shape properties from raw point clouds." Computer Graphics Forum. Vol. 37. No. 2. 2018. [paper]
-
Hermosilla, Pedro, Tobias Ritschel, and Timo Ropinski. "Total Denoising: Unsupervised learning of 3D point cloud cleaning." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Nezhadarya, Ehsan, et al. "Adaptive Hierarchical Down-Sampling for Point Cloud Classification." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
-
Lang, Itai, Asaf Manor, and Shai Avidan. "SampleNet: Differentiable Point Cloud Sampling." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
Generative methods
-
Xiang, Chong, Charles R. Qi, and Bo Li. "Generating 3d adversarial point clouds." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Shu, Dong Wook, Sung Woo Park, and Junseok Kwon. "3d point cloud generative adversarial network based on tree structured graph convolutions." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Zhou, Hang, et al. "DUP-Net: Denoiser and upsampler network for 3D adversarial point clouds defense." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Yu, Lequan, et al. "Pu-net: Point cloud upsampling network." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. [paper]
-
Yifan, Wang, et al. "Patch-based progressive 3d point set upsampling." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Hui, Le, et al. "Progressive Point Cloud Deconvolution Generation Network." Proceedings of the European Conference on Computer Vision (ECCV). (2020). [paper]
-
Yuan, Wentao, et al. "Pcn: Point completion network." 2018 International Conference on 3D Vision (3DV). IEEE, 2018. [paper]
-
Wang, Xiaogang, Marcelo H. Ang Jr, and Gim Hee Lee. "Cascaded Refinement Network for Point Cloud Completion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
-
Huang, Zitian, et al. "PF-Net: Point Fractal Network for 3D Point Cloud Completion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [paper]
-
Xie, Haozhe, et al. "GRNet: Gridding Residual Network for Dense Point Cloud Completion." Proceedings of the European Conference on Computer Vision (ECCV). (2020). [paper]
-
Lan, Ziquan, Zi Jian Yew, and Gim Hee Lee. "Robust Point Cloud Based Reconstruction of Large-Scale Outdoor Scenes." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Li, Ruihui, et al. "Pu-gan: a point cloud upsampling adversarial network." Proceedings of the IEEE International Conference on Computer Vision. 2019. [paper]
-
Sarmad, Muhammad, Hyunjoo Jenny Lee, and Young Min Kim. "Rl-gan-net: A reinforcement learning agent controlled gan network for real-time point cloud shape completion." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. [paper]
-
Wu, Rundi, et al. "Multimodal Shape Completion via Conditional Generative Adversarial Networks." Proceedings of the European Conference on Computer Vision (ECCV). (2020). [paper]