Deep Learning Based on Semantic Segmentation for Three-Dimensional Object Detection from Point Clouds

Liang Zhao; Jie Hu; Han Liu; Yongpeng An; Zongquan Xiong; Yu Wang

doi:10.3788/CJL202148.1710004

Chinese Journal of Lasers, Volume. 48, Issue 17, 1710004(2021)

Deep Learning Based on Semantic Segmentation for Three-Dimensional Object Detection from Point Clouds

Liang Zhao^1,2,3,4, Jie Hu^1,2,3,4、*, Han Liu^1,2,3,4, Yongpeng An^1,2,3,4, Zongquan Xiong^1,2,3,4, and Yu Wang^1,2,3,4

Author Affiliations

¹School of Automotive Engineering, Wuhan University of Technology, Wuhan, Hubei 430070, China

²Hubei Key Laboratory of Advanced Technology for Automotive Components, Wuhan University of Technology, Wuhan, Hubei 430070, China

³Hubei Collaborative Innovation Center for Automotive Components Technology, Wuhan University of Technology, Wuhan, Hubei 430070, China

⁴Hubei Research Center for New Energy & Intelligent Connected Vehicle, Wuhan University of Technology, Wuhan, Hubei 430070, China;

show less

Abstract Get PDF(in Chinese)

Figures & Tables(18)

Fig. 1. Comparison of different FPS algorithms. (a) SegFPS algorithm; (b) traditional FPS algorithm

Download full size

Fig. 2. Framework of the Seg-RCNN

Download full size

Fig. 3. Network structure based on original point cloud algorithm

Download full size

Fig. 4. Structure of the SegNet

Download full size

Fig. 5. Visual detection results of our algorithm on the val split

Download full size

Fig. 6. Unlabeled targets in the KITTI dataset

Download full size

Fig. 7. Detected result of the Pedestrian category

Download full size

Fig. 8. Seg-RCNN online detection based on ROS

Download full size

Fig. 9. Principle of the Voxel-based algorithm

Download full size

Fig. 10. Running time of our algorithm on the val split

Download full size

Table 1. Nomenclature

View table

Table 1. Nomenclature

Abbreviation	Explanation
Seg-RCNN	segmentation based region-convolution neural networks
SegFPS	segmentation classes based further point sampling
SegNet	semantic segmentation network for foreground points
NMS	non-maximum-suppression
Grouping	using keypoints to group features
Bev	bird’s eye view
FPS	further point sampling

Table 2. mAP of different algorithms on the KITTI test set unit: %

View table

Table 2. mAP of different algorithms on the KITTI test set unit: %

Algorithm	Reference	Type	Car-3D			Car-Bev.			Cyclist-3D			Pedestrian-3D
Algorithm	Reference	Type	Easy	Mod	Hard	Easy	Mod	Hard	Easy	Mod	Hard	Easy	Mod	Hard
MV3D^[6]	CVPR 2017	RGB+LiDAR	74.97	63.63	54.00	86.62	78.93	69.80	--	--	--	--	--	--
F-PointNet^[28]	CVPR 2018	RGB+LiDAR	82.19	69.79	60.59	91.17	84.67	74.77	72.27	56.12	49.01	--	--	--
ContFuse^[9]	ECCV 2018	RGB+LiDAR	83.68	68.78	61.67	94.07	85.35	75.88				--	--	--
AVOD-FPN^[7]	IROS 2018	RGB+LiDAR	83.07	71.76	65.73	90.99	84.82	79.62	63.76	50.55	44.93	--	--	--
PointRCNN^[29]	CVPR2019	LiDAR	85.94	75.76	68.32	92.13	87.39	82.72	73.93	59.60	53.59
SECOND^[3]	Sensors 2018	LiDAR	83.34	72.55	65.82	89.39	83.77	78.59	71.33	52.08	45.83	--	--	--
PointPillars^[10]	CVPR 2019	LiDAR	82.58	74.31	68.99	90.07	86.56	82.81	77.10	58.65	51.92	--	--	--
VoxelNet^[2]	arXiv 2017	LiDAR	77.47	65.11	57.73	--	--	--	61.22	48.36	44.37	--	--	--
Ours	--	LiDAR	89.16	79.73	72.28	93.36	89.39	81.93	76.23	60.05	54.37	78.17	63.89	56.73

Table 3. mAP of different algorithms on the val split unit: %

View table

Table 3. mAP of different algorithms on the val split unit: %

Algorithm	Reference	Type	Mod	Easy	Hard
MV3D	CVPR 2017	RGB+LiDAR	62.68	--	--
ContFuse	ECCV 2018	RGB+LiDAR	73.25	--	--
F-PointNet	CVPR 2018	RGB+LiDAR	70.92	--	--
AVOD-FPN^[7]	IROS 2018	RGB+LiDAR	74.44	--	--
PointRCNN^[29]	CVPR 2019	LiDAR	78.63	--	--
STD^[30]	ICCV 2019	LiDAR	79.80	--	--
Ours	--	LiDAR	81.11	91.33	77.49

Table 4. 3D mAP of SegNet with different strategies
View table
Table 4. 3D mAP of SegNet with different strategies
SegNet 1 SegNet 2 3D mAP/ %
√ 78.23
√ 81.11

Table 5. Validation of the SegFPS
View table
Table 5. Validation of the SegFPS
FPS SegFPS and FPS fusionsampling strategy SegFPS 3D mAP /%
√ 78.21
√ 79.01
√ 81.11

Table 6. Running time of the Point-based part of our algorithm
View table
Table 6. Running time of the Point-based part of our algorithm
Name of operation SegFPS Grouping FP
Number of operation 1 6 6
Running time /ms 0.141 32.7 5.2

Table 7. Running time of the Voxel-based algorithm
View table
Table 7. Running time of the Voxel-based algorithm
Conv Running time /ms
Conv 1 (16,16) [1] 1.12
Conv 2 (16,32) [3] 6.40
Conv 3 (32,64) [3] 8.62
Conv 4 (64,128) [3] 10.40

Table 8. Running time of each module in Seg-RCNN
View table
Table 8. Running time of each module in Seg-RCNN
Module Running time /ms
Voxel-based 26.54
Point-based 38.04
SegNet 1.24
NMS 6.50
Others data transfer ~8

Tools

Get Citation

Copy Citation Text

Liang Zhao, Jie Hu, Han Liu, Yongpeng An, Zongquan Xiong, Yu Wang. Deep Learning Based on Semantic Segmentation for Three-Dimensional Object Detection from Point Clouds[J]. Chinese Journal of Lasers, 2021, 48(17): 1710004

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites