Road Falling Objects Detection Algorithm Based on Image and Point Cloud Fusion

Haolin Liang; Huaiyu Cai; Bochong Liu; Yi Wang; Xiaodong Chen

doi:10.3788/LOP213044

Laser & Optoelectronics Progress, Volume. 60, Issue 10, 1010001(2023)

Road Falling Objects Detection Algorithm Based on Image and Point Cloud Fusion

Haolin Liang, Huaiyu Cai^*, Bochong Liu, Yi Wang, and Xiaodong Chen

Key Laboratory of Photoelectric Information, Ministry of Education, School of Precision Instruments and Optoelectronic Engineering, Tianjin University, Tianjin 300072, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(13)

Fig. 1. Flow chart of algorithm for detecting falling objects

Download full size

Fig. 2. Structure diagram of ResNet-50

Download full size

Fig. 3. ResNet-50 down-sampling module optimization. (a) Before optimization; (b) after optimization

Download full size

Fig. 4. Schematic diagram of test vehicle and sensor installation

Download full size

Fig. 5. Examples of experimental data collection

Download full size

Fig. 6. Extraction results of road objects. (a) Road edge extraction; (b) ground point cloud filtering; (c) ground point elimination; (d) point cloud clustering

Download full size

Fig. 7. Mapping result of point cloud region of interest in visual image

Download full size

Fig. 8. Detection result of scattered objects in a picture

Download full size

Fig. 9. Prediction result of scattered object image block

Download full size

Table 1. Main equipment parameters used by the system

View table

Table 1. Main equipment parameters used by the system

Equipment	Model	Parameter
Lidar	Velodyne VLP-32C	Maximum measuring range 200 m Field of view 360°（H）×40°（V，-25°-15°） Vertical resolution minimum 0.33° Horizontal resolution 0.1-0.4°
Camera	DaHua 5131M/CU210	Resolution 1280×1024 Pixel size 4.8 μm×4.8 μm Maximum frame rate 210 frame/s CMOS target surface size 1/2''
GNSS	MG910	Centimeter-level positioning Maximum output frequency 20 Hz
IMU	IMU560	Maximum output frequency 100 Hz
Industrial PC	DT-S2010MB-YH310MC4L	CPU：Intel i7-8700 RAM 16 GB 2 TB mechanical hard disk+128 GB SSD

Table 2. Experimental results of scattered objects detection algorithm

View table

Table 2. Experimental results of scattered objects detection algorithm

Total number of road objects	Total number of falling objects	Predicted number of falling objects	Number of falling objects predicted to be true	Unpredictable number of falling objects	Predicted number of falling objects that are false	Precision /%	Recall /%
783	260	252	239	21	13	94.84	91.92

Table 3. Detection of falling objects of different sizes
View table
Table 3. Detection of falling objects of different sizes
Scattered object size /cm Actual number Number of correctly detected Recall /%
<10 17 6 35.29
10-30 191 181 94.76
>30 52 52 100.00

Table 4. Comparison of prediction results before and after network structure optimization
View table
Table 4. Comparison of prediction results before and after network structure optimization
Evaluation index Before optimization After optimization Improvement
Precision /% 93.41 94.84 1.43
Recall /% 90.90 91.92 1.02

Tools

Get Citation

Copy Citation Text

Haolin Liang, Huaiyu Cai, Bochong Liu, Yi Wang, Xiaodong Chen. Road Falling Objects Detection Algorithm Based on Image and Point Cloud Fusion[J]. Laser & Optoelectronics Progress, 2023, 60(10): 1010001

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites