Advancements in Semantic Segmentation Methods for Large-Scale Point Clouds Based on Deep Learning

Da Ai; Xiaoyang Zhang; Ce Xu; Siyu Qin; Hui Yuan

doi:10.3788/LOP231771

Laser & Optoelectronics Progress, Volume. 61, Issue 12, 1200003(2024)

Advancements in Semantic Segmentation Methods for Large-Scale Point Clouds Based on Deep Learning

Da Ai¹, Xiaoyang Zhang^1、*, Ce Xu¹, Siyu Qin¹, and Hui Yuan²

¹School of Communications and Information Engineering, Xi'an University of Posts & Telecommunications, Xi'an 710121, Shaanxi, China

²School of Control Science and Engineering, Shandong University, Jinan 250100, Shandong, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(14)

Fig. 1. Deep learning-based semantic segmentation method for large-scale point clouds

Download full size

Fig. 2. Chronological overview of indirect-based semantic segmentation methods for large-scale point clouds

Download full size

Fig. 3. Network structure of PolarNet^[43]

Download full size

Fig. 4. Chronological overview of direct-, hybrid-, and weakly supervised-based semantic segmentation methods for large-scale point clouds

Download full size

Fig. 5. Network structure of BAAF-Net^[62]

Download full size

Fig. 6. Network structure of RPVNet^[44]

Download full size

Fig. 7. Network structure of MPRM^[95]

Download full size

Fig. 8. Example of the common datasets for large-scale point cloud semantic segmentation. (a) S3DIS^[98]; (b) ScanNet^[99]; (c) Semantic3D^[100]; (d) SemanticKITTI^[65]; (e) Toronto-3D^[103]; (f) nuScenes^[104]; (g) Paris-Lille-3D^[101]; (h) DALES^[105]; (i) SensatUrban^[106]

Download full size

Table 1. Common datasets for semantic segmentation of large-scale point clouds

View table

Table 1. Common datasets for semantic segmentation of large-scale point clouds

	Dataset	Year	Spatial size	Number of classes	Number of points /10⁶	RGB	Sensor	Highlight
Indoor dataset	S3DIS^［98］	2016	$6 \times 10^{3} m^{2}$	13	273	Yes	Matterport	A composition of colored 3D scanned points of interior areas of large buildings
Indoor dataset	ScanNet^［99］	2017	$1.13 \times 10^{5} m^{2}$	20	242	Yes	RGB-D	RGB-D video datasets for 3D object classification，semantic voxel labeling
Outdoor dataset	Semantic3D^［100］	2017		8	4000	Yes	TLS	High-quality TLS data with higher point density and accuracy compared with other datasets
	SemanticKITTI^［65］	2019	$39.2 \times 10^{3} m$	28	4549	No	MLS	Real-world datasets based on automotive LiDAR that can be used to test autonomous driving
	Toronto-3D^［103］	2020	$1 \times 10^{3} m$	8	78.3	Yes	MLS	Available for autonomous driving and urban high-definition maps
	nuScenes^［104］	2020		23	0.001	Yes	MLS	The first multimodal dataset，containing nighttime and rainy day data，describes object classes，locations，attributes，and scenes
City dataset	ISPRS^［107］	2012		9	1.2	No	ALS	The reference data includes 2D contours of multiple object types
	Paris-Lille-3D^［101］	2018	$1.94 \times 10^{3} m$	50	143	No	MLS	The categories are the most numerous，and for each category vehicles can be categorized according to parked，stopped，or moving
	DublinCity^［102］	2020	$2 \times 10^{6} m^{2}$	13	260	No	ALS	The first highly dense ALS point cloud dataset and provides hierarchical labeling
	Campus3D^［108］	2020	$1.58 \times 10^{6} m^{2}$	24	937.1	Yes	UAV photogrammetry	Photogrammetric point cloud datum dataset for enabling hierarchical understanding of outdoor scenes
	DALES^［105］	2020	$10 \times 10^{6} m^{2}$	8	505	No	ALS	Large aerial LiDAR dataset with 400 times the number of points and 6 times the resolution of comparable datasets
	SensatUrban^［106］	2021	$7.64 \times 10^{6} m^{2}$	13	2847	Yes	UAV photogrammetry	The number of labeled points is 3 times more than the largest photogrammetric dataset

Table 2. Segmentation results of different models on ScanNet dataset

View table

Table 2. Segmentation results of different models on ScanNet dataset

Category	Method	mIoU /%
Multi view-based	TangentConv^［49］	43.8
Pointwise MLP-based	PointNet++^［59］	33.9
Convolution-based	KPConv^［38］	68.4
	PointConv^［70］	55.6
	FG-Net^［73］	68.5
	MSPCNN^［71］	56.8
Weak-supervision-based	SQN（0.1%）^［6］	56.9
	PSD（1%）^［94］	54.7
	MPRM^［95］	41.1
	Liu（0.02%）^［96］	69.1
	Zhang（10%）^［97］	52.0
Hybrid-based	FusionNet^［85］	68.8
Hybrid-based	MVPNet^［89］	64.1

Table 3. Segmentation results of different models on S3DIS dataset

View table

Table 3. Segmentation results of different models on S3DIS dataset

Category	Method	Area 5			6-fold
Category	Method	mIoU /%	OA /%	mAcc /%	mIoU /%	OA /%	mAcc /%
Multi-view-based	VMVF^［46］	65.4	—	—	—	—	—
Multi-view-based	TangentConv^［49］	52.8	82.5	62.2	—	—	—
Voxel-based	SegCloud^［53］	48.9	—	57.4	—	—	—
Voxel-based	VV-Net^［55］	—	—	—	78.2	87.8	—
Pointwise MLP-based	PointNet^［10］	41.1	—	49.0	47.6	78.6	56.2
	PointNet++^［59］	—	—	—	54.5	81.0	—
	PointWeb^［60］	60.3	87.0	66.6	66.7	87.3	76.1
	RandLA-Net^［61］	61.6	86.7	—	70.0	88.0	82.0
	SCF-Net^［52］	—	—	—	71.6	88.4	82.7
	BAAF-Net^［62］	—	—	—	72.2	88.9	83.1
Convolution-based	KPConv^［38］	67.1	—	72.8	70.6	—	—
	PointCNN^［66］	57.3	85.9	63.9	65.4	88.1	75.6
	ConvPoint^［67］	—	—	—	68.2	88.8	—
	MappingConvSeg^［68］	—	—	—	66.8	86.8	—
	DenseKPNet^［72］	68.9	90.8	73.9	71.9	89.3	79.7
	MSPCNN^［71］	—	—	—	67.8	87.3	—
Graph-based	DGCNN^［74］	—	—	—	56.1	84.1	—
	SPG^［78］	58.0	86.3	66.5	62.1	85.5	73.0
	SSP+SPG^［79］	61.7	87.9	68.2	68.4	87.9	78.3
Weak-supervision-based	Xu（10%）^［92］	48.0	—	—	—	—	—
	PSD（1%）^［94］	63.5	—	—	68.0	—	—
	Zhang（10%）^［97］	64.0	—	—	68.1	—	—
Hybrid-based	FusionNet^［85］	67.2	—	72.3	—	—	—
Hybrid-based	SPVAN^［87］	—	—	—	69.7	88.4	80.2

Table 4. Segmentation results of different models on Semantic3D dataset

View table

Table 4. Segmentation results of different models on Semantic3D dataset

Category	Method	mIoU /%	OA /%
Multi-view-based	Tangent Conv^［49］	66.4	89.3
Multi-view-based	SnapNet^［50］	59.1	88.6
Pointwise MLP-based	PointNet++^［59］	63.1	85.7
	RandLA-Net^［61］	77.4	94.8
	SCF-Net^［52］	77.6	94.7
	BAAF-Net^［62］	75.4	94.9
Convolution-based	KPConv^［38］	74.6	92.9
	ConvPoint^［67］	76.5	93.4
	FG-Net^［73］	78.2	—
	DenseKPNet^［72］	77.9	94.9
Graph-based	SPG^［78］	73.2	94.0
Graph-based	GPGAN^［77］	70.8	94.1
Weak supervision-based	PSD（1%）^［94］	75.8	—
	SQN（0.1%）^［6］	72.3	94.8
	Zhang（10%）^［97］	73.3	94.0

Table 5. Segmentation results of different models on SemanticKITTI dataset

View table

Table 5. Segmentation results of different models on SemanticKITTI dataset

Category	Method	mIoU /%
Projection-based	SqueezeSeg^［24］	29.5
	SqueezeSegV2^［33］	39.7
	CENet^［34］	64.7
	RangeNet53++^［35］	52.2
	SqueezeSegV3^［36］	55.9
	KPRNet^［37］	63.1
	MFFNet^［39］	68.6
	SalsaNet^［40］	45.4
	SalsaNext^［41］	59.5
	PolarNet^［43］	54.3
Voxel-based	PVCL^［56］	64.0
Voxel-based	Cylindr3D^［57］	67.8
Pointwise MLP-based	PointNet^［10］	14.6
	PointNet++^［59］	20.1
	RandLA-Net^［61］	53.9
	BAAF-Net^［62］	59.9
Convolution-based	KPConv^［38］	58.8
Convolution-based	FG-Net^［73］	53.8
Weak supervision-based	SQN（0.1%）^［6］	50.8
Hybrid-based	SPVAN^［87］	60.8
	SPVNAS^［88］	66.4
	AMVNet^［90］	65.3
	TORNADO-Net^［91］	63.1
	RPVNet^［44］	70.3

Table 6. Segmentation results of different models on Paris-Lille-3D dataset and nuScenes dataset

View table

Table 6. Segmentation results of different models on Paris-Lille-3D dataset and nuScenes dataset

Category	Method	MIoU /%
Category	Method	Paris-Lille-3D dataset	nuScenes dataset
Projection-based	SqueezeSegV2^［33］	36.9	—
	RangeNet53++^［35］	—	65.5
	SalsaNext^［41］	—	72.2
	PolarNet ^［43］	43.7	71.0
Multi-view-based	LIF-Seg^［48］	—	78.2
Voxel-based	PVCL^［56］	—	73.9
Voxel-based	Cylindr3D^［57］	—	76.1
Pointwise MLP-based	PointNet^［10］	38.6	—
Pointwise MLP-based	PointNet++^［59］	32.9	—
Convolution-based	KPConv^［38］	82.0	—
	ConvPoint^［67］	75.9	—
	FG-Net^［73］	82.3	—
	MSPCNN^［71］	70.5	—
Graph-based	DGCNN^［74］	52.9	—
Graph-based	GPGAN^［77］	80.3	—
Hybrid-based	SPVNAS^［87］	—	77.4
	AMVNet^［90］	—	76.1
	RPVNet^［44］	—	77.6

Tools

Get Citation

Copy Citation Text

Da Ai, Xiaoyang Zhang, Ce Xu, Siyu Qin, Hui Yuan. Advancements in Semantic Segmentation Methods for Large-Scale Point Clouds Based on Deep Learning[J]. Laser & Optoelectronics Progress, 2024, 61(12): 1200003

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Reviews

Received: Jul. 21, 2023

Accepted: Sep. 18, 2023

Published Online: Jun. 5, 2024

The Author Email: Xiaoyang Zhang (zxy1017254139@163.com)

DOI:10.3788/LOP231771

CSTR:32186.14.LOP231771

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology