Chinese Journal of Lasers, Volume. 51, Issue 5, 0509001(2024)

Review of 3D Point Cloud Processing Methods Based on Deep Learning

Yiquan Wu*, Huixian Chen, and Yao Zhang
Author Affiliations
  • College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, Jiangsu, China
  • show less
    References(128)

    [1] Zheng S W, Li W H, Hu J Y. Vehicle detection in the traffic environment based on the fusion of laser point cloud and image information[J]. Chinese Journal of Scientific Instrument, 40, 143-151(2019).

    [2] Yu H S, Fu Q, Sun J et al. Improved 3D-NDT point cloud registration algorithm for indoor mobile robot[J]. Chinese Journal of Scientific Instrument, 40, 151-161(2019).

    [3] Espinosa N, Lenz A, Gross W et al. Towards fast 3D reconstruction of urban areas from aerial nadir images for a near real-time remote sensing system[C], 6472-6475(2018).

    [4] Zhang Z J, Cheng X J, Cao Y J et al. Application of 3D reconstruction of relic sites combined with laser and vision point cloud[J]. Chinese Journal of Lasers, 47, 1110001(2020).

    [5] Shang Y H, Zhang G Q, Meng Z J et al. Field obstacle detection method of 3D LiDAR point cloud based on Euclidean clustering[J]. Transactions of the Chinese Society for Agricultural Machinery, 53, 23-32(2022).

    [6] Zhang B, Dai N, Tian S K et al. Extraction of tooth preparation line based on sparse octree convolutional neural network[J]. Journal of Computer-Aided Design & Computer Graphics, 31, 2129-2135(2019).

    [7] Ma W F, Wang C, Wang J L et al. Shape simulation for a transmission line under different temperatures based on point cloud data[J]. Power System Protection and Control, 49, 90-96(2021).

    [8] Gu F, Zhang C L, Guo Z G et al. Combined filtering algorithm for extracting bridge point cloud[J]. Laser & Optoelectronics Progress, 60, 0211001(2023).

    [9] Li W G, Mei Y, Fan X et al. Railway track detection based on vehicle laser point cloud[J]. Chinese Journal of Lasers, 49, 0410002(2022).

    [10] Liu W Q, Lai B Q, Wang C et al. Learning to match 2D images and 3D LiDAR point clouds for outdoor augmented reality[C], 654-655(2020).

    [11] Wirth F, Quehl J, Ota J et al. PointAtMe: efficient 3D point cloud labeling in virtual reality[C], 1693-1698(2019).

    [12] Xie Z X, Li M H. A survey on machine learning in recognition of 3D object based on point cloud[J]. Periodical of Ocean University of China, 51, 125-130, 98(2021).

    [13] Deng L, Yu D. Deep learning: methods and applications[J]. Foundations and Trends® in Signal Processing, 7, 197-387(2014).

    [14] Zhou L, Sun G X, Li Y et al. Point cloud denoising review: from classical to deep learning-based approaches[J]. Graphical Models, 121, 101140(2022).

    [15] Zhang H R, Dong Z, Yang B S et al. Research progress and trend of point cloud compression[J]. Geomatics and Information Science of Wuhan University, 48, 192-205(2023).

    [16] Bi Y, Pan M Q, Zhang S et al. Overview of 3D point cloud super-resolution technology[J]. Chinese Optics, 15, 210-223(2022).

    [17] Liu C X, Wei M Q, Guo Y W. 3D point cloud restoration via deep learning: a comprehensive survey[J]. Journal of Computer-Aided Design & Computer Graphics, 33, 1936-1952(2021).

    [18] Roveri R, Öztireli A C, Pandele I et al. PointProNets: consolidation of point clouds with convolutional neural networks[J]. Computer Graphics Forum, 37, 87-99(2018).

    [19] Pistilli F, Fracastoro G, Valsesia D et al. Learning robust graph-convolutional representations for point cloud denoising[J]. IEEE Journal of Selected Topics in Signal Processing, 15, 402-414(2021).

    [20] Chen Z W, Li P, Wei Z Y et al. GeoGCN: geometric dual-domain graph convolution network for point cloud denoising[C](2023).

    [21] Lu D N, Lu X Q, Sun Y X et al. Deep feature-preserving normal estimation for point cloud filtering[J]. Computer-Aided Design, 125, 102860(2020).

    [22] Jang J W, Kwon Y C, Lim H et al. CNN-based denoising, completion, and prediction of whole-body human-depth images[J]. IEEE Access, 7, 175842-175856(2019).

    [23] Heinzler R, Piewak F, Schindler P et al. CNN-based lidar point cloud de-noising in adverse weather[J]. IEEE Robotics and Automation Letters, 5, 2514-2521(2020).

    [24] Seppänen A, Ojala R, Tammi K. 4DenoiseNet: adverse weather denoising from adjacent point clouds[J]. IEEE Robotics and Automation Letters, 8, 456-463(2022).

    [25] Zhou H, Chen K J, Zhang W M et al. DUP-Net: denoiser and upsampler network for 3D adversarial point clouds defense[C], 1961-1970(2020).

    [26] Liu X H, Liu X C, Liu Y S et al. SPU-Net: self-supervised point cloud upsampling by coarse-to-fine reconstruction with self-projection optimization[J]. IEEE Transactions on Image Processing, 31, 4213-4226(2022).

    [27] Qian Y, Hou J H, Kwong S, Vedaldi A, Bischof H, Brox T et al. PUGeo-Net: a geometry-centric network for 3D point cloud upsampling[M]. Computer vision-ECCV 2020, 12364, 752-769(2020).

    [28] Qian Y, Hou J H, Kwong S et al. Deep magnification-flexible upsampling over 3D point clouds[J]. IEEE Transactions on Image Processing, 30, 8354-8367(2021).

    [29] Long C, Zhang W X, Li R H et al. PC2-PU: patch correlation and point correlation for effective point cloud upsampling[C], 2191-2201(2022).

    [30] Han B, Zhang X Y, Ren S. PU-GACNet: graph attention convolution network for point cloud upsampling[J]. Image and Vision Computing, 118, 104371(2022).

    [31] Yu L Q, Li X Z, Fu C W, Ferrari V, Hebert M, Sminchisescu C et al. EC-Net: an edge-aware point set consolidation network[M]. Computer vision-ECCV 2018, 11211, 398-414(2018).

    [32] Duan C J, Chen S H, Kovacevic J. 3D point cloud denoising via deep neural network based local surface estimation[C], 8553-8557(2019).

    [33] Casajus P H, Ritschel T, Ropinski T. Total denoising: unsupervised learning of 3D point cloud cleaning[C]. Republic of Korea, 52-60(2019).

    [34] Guerrero P, Kleiman Y, Ovsjanikov M et al. PCPNet learning local shape properties from raw point clouds[J]. Computer Graphics Forum, 37, 75-85(2018).

    [35] Rakotosaona M J, la Barbera V, Guerrero P et al. PointCleanNet: learning to denoise and remove outliers from dense point clouds[J]. Computer Graphics Forum, 39, 185-203(2020).

    [36] Zhang D B, Lu X Q, Qin H et al. Pointfilter: point cloud filtering via encoder-decoder modeling[J]. IEEE Transactions on Visualization and Computer Graphics, 27, 2015-2027(2021).

    [38] Wang X T, Fan X P, Zhao D B. PointFilterNet: a filtering network for point cloud denoising[J]. IEEE Transactions on Circuits and Systems for Video Technology, 33, 1276-1290(2023).

    [39] Luo S T, Hu W. Score-based point cloud denoising[C], 4563-4572(2022).

    [40] Chen H L, Du B A, Luo S T et al. Deep point set resampling via gradient fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 2913-2930(2023).

    [43] Xu X W. Research on denoising algorithm of 3D point cloud based on deep learning[D], 15-22(2022).

    [44] Zhu X S, Ma S, Chen D X et al. MSaD-Net: a mix self-attention networks for 3D point cloud denoising[J]. IEEE Photonics Journal, 15, 8500607(2023).

    [45] Chen H H, Wei Z Y, Li X Z et al. RePCD-Net: feature-aware recurrent point cloud denoising network[J]. International Journal of Computer Vision, 130, 615-629(2022).

    [46] Luo S T, Hu W. Differentiable manifold reconstruction for point cloud denoising[C], 1330-1338(2020).

    [48] Regaya Y, Fadli F, Amira A. Point-Denoise: unsupervised outlier detection for 3D point clouds enhancement[J]. Multimedia Tools and Applications, 80, 28161-28177(2021).

    [50] Huang L L, Wang S L, Wong K et al. OctSqueeze: octree-structured entropy model for LiDAR compression[C], 1310-1320(2020).

    [51] Fu C Y, Li G, Song R et al. OctAttention: octree-based large-scale contexts model for point cloud compression[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 625-633(2022).

    [52] Que Z Z, Lu G, Xu D. VoxelContext-Net: an octree based framework for point cloud compression[C], 6038-6047(2021).

    [53] Nguyen D T, Quach M, Valenzise G et al. Learning-based lossless compression of 3D point cloud geometry[C], 4220-4224(2021).

    [54] Nguyen D T, Quach M, Valenzise G et al. Multiscale deep context modeling for lossless point cloud geometry compression[C](2021).

    [56] Wang J Q, Ding D D, Ma Z. Lossless point cloud attribute compression using cross-scale, cross-group, and cross-color prediction[C], 228-237(2023).

    [57] Quach M, Valenzise G, Dufaux F. Improved deep point cloud geometry compression[C](2020).

    [58] Wang J Q, Ding D D, Li Z et al. Multiscale point cloud geometry compression[C], 73-82(2021).

    [61] Quach M, Valenzise G, Dufaux F. Learning convolutional transforms for lossy point cloud geometry compression[C], 4320-4324(2019).

    [62] Guarda A F R, Rodrigues N M M, Pereira F. Point cloud coding: adopting a deep learning-based approach[C](2020).

    [63] Huang T X, Liu Y. 3D point cloud geometry compression on deep learning[C], 890-898(2019).

    [64] He Y, Ren X L, Tang D H et al. Density-preserving deep point cloud compression[C], 2323-2332(2022).

    [65] Wen X Z, Wang X, Hou J H et al. Lossy geometry compression of 3D point cloud data via an adaptive octree-guided network[C](2020).

    [66] Milani S. A syndrome-based autoencoder for point cloud geometry compression[C], 2686-2690(2020).

    [67] Feng Y, Liu S S, Zhu Y H. Real-time spatio-temporal LiDAR point cloud compression[C], 10766-10773(2021).

    [68] Sheng X H, Li L, Liu D et al. Deep-PCAC: an end-to-end deep lossy compression framework for point cloud attributes[J]. IEEE Transactions on Multimedia, 24, 2617-2632(2022).

    [69] Quach M, Valenzise G, Dufaux F. Folding-based compression of point cloud attributes[C], 3309-3313(2020).

    [70] Zhao L L, Ma K K, Lin X H et al. Real-time LiDAR point cloud compression using bi-directional prediction and range-adaptive floating-point coding[J]. IEEE Transactions on Broadcasting, 68, 620-635(2022).

    [71] Zhao Z, Ma Y X, Xu K et al. An optimized scalable binary quantization network for lidar point cloud learning[J]. Acta Optica Sinica, 42, 1212005(2022).

    [72] Wang J Q, Ding D D, Li Z et al. Sparse tensor-based multiscale representation for point cloud geometry compression[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 9055-9071(2023).

    [73] Liang Z J, Liang F. TransPCC: towards deep point cloud compression via Transformers[C](2022).

    [74] Liu G X, Wang J Q, Ding D D et al. PCGFormer: lossy point cloud geometry compression via local self-attention[C](2023).

    [76] Yu L Q, Li X Z, Fu C W et al. PU-Net: point cloud upsampling network[C], 2790-2799(2018).

    [77] Meng H, Han Y, Chen J et al. A point cloud density enhancement method based on super-resolution convolutional neural network[C], 8-12(2020).

    [78] Yue J, Wen W S, Han J et al. 3D point clouds data super resolution-aided LiDAR odometry for vehicular positioning in urban canyons[J]. IEEE Transactions on Vehicular Technology, 70, 4098-4112(2021).

    [79] Wang G Y, Xu G, Wu Q et al. Two-stage point cloud super resolution with local interpolation and readjustment via outer-product neural network[J]. Journal of Systems Science and Complexity, 34, 68-82(2021).

    [80] Chen K, Liu C, Ou Y S. Channel attention based network for LiDAR super-resolution[C], 5458-5463(2022).

    [81] Matsuzaki K, Komorita S. Efficient deep super-resolution of voxelized point cloud in geometry compression[J]. IEEE Sensors Journal, 23, 1328-1342(2022).

    [82] Qian G C, Abualshour A, Li G H et al. PU-GCN: point cloud upsampling using graph convolutional networks[C], 11678-11687(2021).

    [83] Zhong F, Bai Z Y. 3D point cloud super-resolution using dynamic residual graph convolution[J]. Journal of Zhejiang University (Engineering Science), 56, 2251-2259(2022).

    [84] Zhou Y C, Zhang X F, Wang S S et al. Multi-attribute joint point cloud super-resolution with adversarial feature graph networks[C](2022).

    [85] Li R H, Li X Z, Fu C W et al. PU-GAN: a point cloud upsampling adversarial network[C], 7202-7211(2020).

    [87] Zhou K Y, Dong M, Arslanturk S. “Zero-Shot” point cloud upsampling[C](2022).

    [88] Wang Y F, Wu S H, Huang H et al. Patch-based progressive 3D point set upsampling[C], 5951-5960(2020).

    [89] Li J X, Zhu F Y, Yang X et al. 3D face point cloud super-resolution network[C](2021).

    [90] Qiu S, Anwar S, Barnes N, Wang L, Gall J, Chin T J et al. PU-transformer: point cloud upsampling transformer[M]. Computer vision-ACCV 2022, 13841, 326-343(2022).

    [91] Wang S H, Wang H Q, She S F et al. Swin-T-NFC CRFs: an encoder-decoder neural model for high-precision UAV positioning via point cloud super resolution and image semantic segmentation[J]. Computer Communications, 197, 52-60(2023).

    [92] Charles R Q, Hao S, Mo K C et al. PointNet: deep learning on point sets for 3D classification and segmentation[C], 77-85(2017).

    [93] Li Y X, Deng H P, Xiang S et al. Depth map super-resolution reconstruction based on the texture edge-guided approach[J]. Journal of Image and Graphics, 23, 1508-1517(2018).

    [94] Nguyen D, Choi S, Kim W et al. Graph X-convolution for point cloud deformation in 2D-to-3D conversion[C], 8627-8636(2020).

    [95] Choi S, Nguyen A D, Kim J et al. Point cloud deformation for single image 3D reconstruction[C], 2379-2383(2019).

    [96] Guo C, Dai N, Tian S K et al. Morphological design of missing tooth driven by high-resolution deep generation network[J]. Journal of Image and Graphics, 25, 2249-2258(2020).

    [97] Zhang W T, Jiang H L, Yang Z et al. Data-driven upsampling of point clouds[J]. Computer-Aided Design, 112, 1-13(2019).

    [99] Wang S Q, Zhang J Q, Li L Y et al. Application of MVSNet in 3D reconstruction of space objects[J]. Chinese Journal of Lasers, 49, 2310003(2022).

    [100] Yuan W T, Khot T, Held D et al. PCN: point completion network[C], 728-737(2018).

    [101] Tchapmi L P, Kosaraju V, Rezatofighi H et al. TopNet: structural point cloud decoder[C], 383-392(2020).

    [102] Huang H H, Chen H L, Li J X. Deep neural network for 3D point cloud completion with multistage loss function[C], 4604-4609(2019).

    [103] Huang Z T, Yu Y K, Xu J W et al. PF-Net: point fractal network for 3D point cloud completion[C], 7659-7667(2020).

    [104] Wang X G, Ang M H, Lee G H. Cascaded refinement network for point cloud completion[C], 787-796(2020).

    [105] Wen X, Li T Y, Han Z Z et al. Point cloud completion by skip-attention network with hierarchical folding[C], 1936-1945(2020).

    [106] Long J N, Zhu Q M, He H et al. 3D point cloud completion based on multi-scale degradation[C](2023).

    [107] Wu H, Miao Y B. Cross-regional attention network for point cloud completion[C], 10274-10280(2021).

    [108] Li J, Guo S W, Meng X T et al. DPG-Net: densely progressive-growing network for point cloud completion[J]. Neurocomputing, 491, 1-13(2022).

    [109] Liu X P, Ma Y X, Xu K et al. AGFA-Net: adaptive global feature augmentation network for point cloud completion[J]. IEEE Geoscience and Remote Sensing Letters, 19, 7004505(2022).

    [110] Liu X P, Ma Y X, Xu K et al. Multi-scale Transformer based point cloud completion network[J]. Journal of Image and Graphics, 27, 538-549(2022).

    [113] Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C], 3354-3361(2012).

    [114] Serna A, Marcotegui B, Goulette F et al. Paris-rue-madame database: a 3D mobile laser scanner dataset for benchmarking urban detection, segmentation and classification methods[C](2014).

    [115] Lian Z, Zhang J, Choi S et al. Non-rigid 3D shape retrieval[C], 107-120(2015).

    [116] Wu Z R, Song S R, Khosla A et al. 3D ShapeNets: a deep representation for volumetric shapes[C], 1912-1920(2015).

    [118] Gaidon A, Wang Q, Cabon Y et al. Virtual worlds as proxy for multi-object tracking analysis[C], 4340-4349(2016).

    [119] Yi L, Kim V G, Ceylan D et al. A scalable active framework for region annotation in 3D shape collections[J]. ACM Transactions on Graphics, 35, 210(2016).

    [120] Armeni I, Sener O, Zamir A R et al. 3D semantic parsing of large-scale indoor spaces[C], 1534-1543(2016).

    [121] Zeng A, Song S R, Nießner M et al. 3DMatch: learning local geometric descriptors from RGB-D reconstructions[C], 199-208(2017).

    [122] Dai A, Chang A X, Savva M et al. ScanNet: richly-annotated 3D reconstructions of indoor scenes[C], 2432-2443(2017).

    [123] Chang A, Dai A, Funkhouser T et al. Matterport3[C], 667-676(2018).

    [124] Behley J, Garbade M, Milioto A et al. SemanticKITTI: a dataset for semantic scene understanding of LiDAR sequences[C], 9296-9306(2020).

    [125] Schwarz S, Preda M, Baroncini V et al. Emerging MPEG standards for point cloud compression[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9, 133-148(2018).

    [126] Caesar H, Bankiti V, Lang A H et al. nuScenes: a multimodal dataset for autonomous driving[C], 11618-11628(2020).

    [127] Sun P, Kretzschmar H, Dotiwalla X et al. Scalability in perception for autonomous driving: Waymo open dataset[C], 2443-2451(2020).

    [128] Graziosi D, Nakagami O, Kuma S et al. An overview of ongoing point cloud compression standardization activities: video-based (V-PCC) and geometry-based (G-PCC)[J]. APSIPA Transactions on Signal and Information Processing, 9, 1-17(2020).

    Tools

    Get Citation

    Copy Citation Text

    Yiquan Wu, Huixian Chen, Yao Zhang. Review of 3D Point Cloud Processing Methods Based on Deep Learning[J]. Chinese Journal of Lasers, 2024, 51(5): 0509001

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: holography and information processing

    Received: Jun. 19, 2023

    Accepted: Aug. 11, 2023

    Published Online: Mar. 1, 2024

    The Author Email: Wu Yiquan (nuaaimage@163.com)

    DOI:10.3788/CJL230924

    CSTR:32183.14.CJL230924

    Topics