Infrared and Laser Engineering, Volume. 52, Issue 8, 20230427(2023)

Advancements in fusion calibration technology of lidar and camera

Shiqiang Wang, Zhaozong Meng, Nan Gao, and Zonghua Zhang
Author Affiliations
  • School of Mechanical Engineering, Hebei University of Technology, Tianjin 300130, China
  • show less
    References(100)

    [1] X Q Xu. Survey of multi-sensor data fusion target recognition algorithms. Infrared and Laser Engineering, 35, 321-328(2006).

    [2] [2] Liu Y P, Guo X J, Su S J, et al. Review of a 3D lidar combined with single vision calibration[C]IEEE International Conference on Data Science Computer Application (ICDSCA), 2021, 2021: 397404.

    [3] [3] Kassir A, Peynot T. Reliable automatic cameralaser calibration[C]Australian Robotics Automation Association (ARAA), 2010: 110.

    [5] [5] Stamos I. Automated registration of 3Drange with 2Dcol images: an overview[C]44th Annual Conference on Infmation Sciences Systems (CISS), 2010: 16.

    [8] J Zhang, W T Jiang. Registration between laser scanning point cloud and optical images: status and trends. Journal of Geo-information Science, 19, 528-539(2017).

    [10] [10] Mei C, Rives P. Single view point omnidirectional camera calibration from planar grids[C]IEEE International Conference on Robotics Automation (ICRA), 2007: 39453950.

    [11] X R Yang, X W Zhang, S Y Cheng, et al. Progress on camera calibration methods in visual measurement. Machinery Design and Manufacture, 217, 259-261(2009).

    [12] Y I Abdel-aziz, H M Karara. Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. Photogrammetric Engineering and Remote Sensing, 5, 103-107(2015).

    [13] [13] Tsai R Y. An efficient accurate camera calibration technique f 3D machine vision[C]IEEE Conference of Computer Vision Pattern Recognition(ICCV), 1986: 364374.

    [15] [15] Faugeras O D, Luong Q T, Maybank S J. Camera selfcalibration: they experiments[C]European Conference on Computer Vision(ECCV), 1992: 321334.

    [16] [16] Beardsley P, Murray D. Camera calibration using vanishing points[C]Proceedings of the British Machine Vision Conference (BMVC), 1992: 416425.

    [19] X L Zhang, K Jiang, K Sun, et al. Intrinsic parameters modeling for LiDAR and point cloud correction method. Automotive Engineering, 43, 1228-1237, 1253(2021).

    [20] D Y Qiao, W Z Yuan, Y Ren. Review of MEMS LiDAR. Microelectronics and Computer, 40, 41-49(2023).

    [22] Z Zhu, J L Liu. Calibration of a multi-beam LIDAR by using linear instrumental error model. Applied Mechanics and Materials, 380, 911-914(2013).

    [23] [23] Muhammad N, Lacroix S. Calibration of a rotating multibeam lidar[C]IEEERSJ International Conference on Intelligent Robots Systems(IROS), 2010: 56485653.

    [24] [24] Bergelt R, Khan O, Hardt W. Improving the intrinsic calibration of a velodyne LiDAR sens[C]IEEE Senss, 2017: 13.

    [25] L Zalud, P Kocmanova, F Burian, et al. Calibration and evaluation of parameters in a 3D proximity rotating scanner. Elektronir Elektrotechnika, 21, 3-12(2015).

    [27] J Levinson, S Thrun. Unsupervised calibration for multi-beam lasers. Experimental Robotics, 1, 179-193(2014).

    [29] B S Yang, F X Liang, R G Huang. Progress, challenges and perspectives of 3D lidar point cloud processing. Acta Geodaetica et Cartographica Sinica, 46, 1509-1516(2017).

    [31] [31] Rusu R B, Cousins S. 3D is here: point cloud library (PCL)[C]IEEE International Conference on Robotics Automation (ICRA), 2011: 14.

    [33] L Chen, Y Cai, J S Zhang. Adaptive k-means clustering simplification of scattered point cloud. Journal of Image and Graphics, 22, 1089-1097(2017).

    [34] Y H Zhang, G H Geng, X R Wei, et al. Feature extraction of point clouds using the DBSCAN clustering. Journal of Xidian University, 44, 114-120(2017).

    [35] Z L Sun, Z R Zhou, A Q Wu. An improved mean shift algorithm used for point cloud dat a filtering. Engineering of Surveying and Mapping, 20, 57-59(2011).

    [36] Z Y Wang, Y M Zhang, Y B Chen, et al. Optimization of indoor scene semantic segmentation network based on RGB-D image. Automation and Information Engineering, 41, 27-32(2020).

    [37] V Chernozhukov, D Chetverikov, M Demirer, et al. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21, C1-C68(2017).

    [41] C L Yuan, Z L Xiong, X H Zhou, et al. Study of infrared image edge detection based on sobel operator. Laser and Infrared, 39, 85-87(2009).

    [42] J S Li, Y Ma, F Z Zhao, et al. A novel arithmetic of image edge detection of canny operator. Acta Photonica Sinica, 40, 50-54(2011).

    [43] M Y Xiao, X F Li, X L Zhang, et al. Medical image segmentation algorithm based on multi-scale region growing. Journal of Jilin University, 47, 1591-1597(2017).

    [44] G Q Wang, X H Zhou, L L Yu. Image segmentation based on watershed algorithm. Computer Simulation, 26, 255-258(2009).

    [45] [45] Zhang Q L, Pless R. Extrinsic calibration of a camera laser range finder (improves camera calibration)[C]\\IEEERSJ International Conference on Intelligent Robots Systems(IROS), 2004: 23012306.

    [46] [46] Unnikrishnan R, Hebert M. Fast extrinsic calibration of a laser rangefinder to a camera[D]. Pittsburgh: Carnegie Mellon University, 2005.

    [47] [47] Geiger A Moosmann F, Car O, et al. Automatic camera range sens calibration using a single shot[C]IEEE International Conference on Robotics Automation (ICRA), 2012: 1418.

    [48] Y Chen, Z Chen, S G Wei. Research on point cloud data segmentation based on extrinsic calibration of laser and CCD. Microcomputer Information, 24, 295-297(2008).

    [49] Z Y Han, C G Pu, C Liu. Extrinsic parameters calibration method for pinhole camera and 3D LiDAR. Transducer and Microsystem Technologies, 37, 9-12, 16(2018).

    [50] K Q Wei, Z F Shi, H Li, et al. Research on the joint calibration of monocular camera and 3D Lidar. Navigation Positioning and Timing, 2, 69-74(2015).

    [51] L Li, X Zhang, D W Tu. Joint calibration of 2D and 3D vision integrated sensor system. Chinese Journal of Scientific Instrument, 33, 2473-2479(2012).

    [52] Z Y Xiang, L Zheng. Novel joint calibration method of camera and 3D laser range finder. Journal of Zhejiang University(Engineering Science), 43, 1401-1405(2009).

    [53] H S Li, X N Luo, C G Deng, et al. Joint calibration of sports camera and lidar based on LM algorithm. Journal of Guilin University of Electronic Technology, 42, 345-353(2022).

    [54] G H Kang, Q Zhang, H Zhang, et al. Joint calibration of camera and lidar based on point cloud center. Chinese Journal of Scientific Instrument, 40, 118-126(2019).

    [56] F X Dong, J Cai, Y M Xie. Joint calibration method for stereo vision system and 3D laser system. Chinese Journal of Scientific Instrument, 38, 2589-2596(2017).

    [57] D Q Yu, G Y Li, L Wang, et al. Calibration of LiDAR and camera based on 3D feature point sets. Bulletin of Surveying and Mapping, 11, 40-45(2018).

    [60] [60] Debattisti S, Mazzei L, Panciroli M. Automated extrinsic laser camera intercalibration using triangular targets[C]IEEE Intelligent Vehicles Symposium (IV), 2013: 696–701.

    [61] Z Q Huang, Y Su, Q W Wang, et al. Research on extrinsic parameter calibration method of 2D laser rangefinder (LRF) and visible light camera. Chinese Journal of Scientific Instrument, 41, 121-129(2020).

    [62] Z H Deng, M Y Liu, M Y Fu. An improved method of matching point extracting for vision sensor and LMS space alignment. Optical Technique, 36, 43-47(2010).

    [65] [65] Pusztai Z, Hajder L. Accurate calibration of LiDARcamera systems using dinary boxes[C]IEEE International Conference on Computer Vision Wkshops (ICCV), 2017: 394402.

    [67] F Zhu, J Fan, Y C Huang, et al. Extrinsic calibration of camera and 2D laser-rangefinder with various chessboard constrains. Geomatics and Information Science of Wuhan University, 44, 1524-1529, 1537(2019).

    [69] [69] Toth T, Pusztai Z, Hajder L. Automatic LiDARcamera calibration of extrinsic parameters using a spherical target[C]IEEE International Conference on Robotics Automation (ICRA), 2020: 85808586.

    [70] [70] Liu C, Jiang Y, Yang M. Joint calibration of a camera a laser range finder from grid calibration target[C]IEEE International Conference on Cyber Technology in Automation, Control, Intelligent Systems (CYBER), 2016: 497501.

    [71] [71] Moghadam P, Bosse M, Zlot R. Linebased extrinsic calibration of range image senss[C]\\IEEE International Conference on Robotics Automation(ICRA), 2013: 6–10.

    [72] [72] Scaramuzza D, Harati A, Siegwart R. Extrinsic selfcalibration of a camera a 3D laser range finder from natural scenes[C]IEEERSJ International Conference on Intelligent Robots Systems (IROS), 2007: 41644169.

    [73] [73] GomezOjeda R, Briales J, FernezMal E, et al. Extrinsic calibration of a 2D laserrangefinder a camera based on scene cners[C]IEEE International Conference on Robotics Automation (ICRA), 2015: 36113616.

    [75] J Kang, N L Doh. Automatic targetless camera-LiDAR calibration by aligning edge with Gaussian mixture model. Journal of Field Robotics, 37, 25-37(2020).

    [76] [76] Levinson J, Thrun S. Automatic online calibration of cameras lasers[MOL]. (20130624)[2023027]. https:rss2013.robotics.tuberlin.deimagesbooklet.online.pdf.

    [78] X Y Liu, C J Yuan, F Zhang. Targetless Extrinsic Calibration of Multiple Small FoV LiDARs and Cameras Using Adaptive Voxelization. IEEE Transactions on Instrumentation and Measurement, 71, 1-12(2022).

    [79] [79] Wang W M, Nobuhara S, Nakamura R, et al. SOIC: semantic online initialization calibration f LiDAR camera[EBOL]. (20200309)[20230615]. https:arxiv.gabs2003.04260.

    [80] [80] Abedini A. An investigation into the registration of LiDAR intensity data aerial images using the SIFT approach[C]The International Archives of The Photoghrammetry, Remote Sensing Spatial Infmation Sciences(ISPRS), 2008: 169–176.

    [81] [81] Pey G, McBride J, Savarese S, et al. Automatic targetless extrinsic calibration of a 3D LiDAR camera by maximizing mutual infmation[C]AAAI Conference on Artificial Intelligence (AAAI), 2012, 26(1): 20532059.

    [83] [83] Tayl Z, Nieto J. A mutual infmation approach to automatic calibration of camera lidar in natural environments[C]Australian Conference on Robotics Automation (ACRA), 2012: 39.

    [84] W S Song, Z H Zhang, N Gao, et al. Spatial pose calibration method for lidar and camera based on intensity information. Laser & Optoelectronics Progress, 59, 356-364(2022).

    [85] [85] Huang K H, Stachniss C. Extrinsic multisens calibration f mobile robots using the GaussHelmert model[C]IEEE, RSJ International Conference on Intelligent Robots Systems (IROS), 2017: 19701976.

    [86] [86] Tayl Z, Nieto J. Motionbased calibration of multi modal sens arrays[C]\\IEEE International Conference on Robotics Automation (ICRA), 2015: 48434850.

    [87] [87] Ishikawa R, Oishi T, Ikeuchi K. LiDAR camera calibration using motion estimated by sens fusion odometry[C]IEEERSJ International Conference on Intelligent Robots Systems (IROS), 2018: 73427349.

    [89] C Yang, T J Li, H Hu, et al. Online lidar calibration algorithm based on nonlinear optimization. Computer Engineering and Design, 42, 1919-1925(2021).

    [90] [90] Nedevschi S. Online crosscalibration of camera lidar[C]IEEE International Conference on Intelligent Computer Communication Processing (ICCP), 2017: 295301.

    [91] [91] Schneider N, Piewak F, Stiller C, et al. Reg: multimodal sens registration using deep neural wks[C]IEEE Intelligent Vehicles Symposium (IV), 2017: 18031810.

    [92] [92] Iyer G, Ram R K, Murthy J K, et al. Calib: Geometrically supervised extrinsic calibration using 3D spatial transfmer wks[C]IEEERSJ International Conference on Intelligent Robots Systems (IROS), 2018: 11101117.

    [94] [94] Lv X D, Wang B, Dou Z, et al. LCC: LiDAR camera selfcalibration using cost volume wk[C]IEEE Conference on Computer Vision Pattern Recognition Wkshops (CVPR), 2021: 28882895.

    [95] [95] Cattaneo D, Vaghi M, Ballardini A L, at el. CMR: Camera to LiDARmap registration[C]IEEE Intelligent Transptation Systems Conference(ITSC), 2019: 12831289.

    [96] [96] Shi J Y, Zhu Z H, Zhang J H, et al. CalibRCNN: Calibrating camera LiDAR by recurrent convolutional neural wk geometric constraints[C]IEEE, RSJ International Conference on Intelligent Robots Systems(IROS), 2020: 12831289.

    [97] [97] Zhao G N, Hu J S, You S Y, et al. CalibDNN: multimodal sens calibration f perception using deep neural wks[C]Conference on Signal Processing, SensInfmation Fusion, Target Recognition, 2021: 324335.

    [99] [99] Jing X, Ding X Q, Xiong R, et al. DXQ: Differentiable LiDARcamera extrinsic calibration using qualityaware flow[C]International Conference on Intelligent Robots Systems(IROS), 2022: 62356241.

    [100] W Yi, M Zhu, J Liang. PSNet: LiDAR and camera registration using parallel Subnetworks. IEEE Access, 10, 70553-70561(2022).

    Tools

    Get Citation

    Copy Citation Text

    Shiqiang Wang, Zhaozong Meng, Nan Gao, Zonghua Zhang. Advancements in fusion calibration technology of lidar and camera[J]. Infrared and Laser Engineering, 2023, 52(8): 20230427

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Jun. 15, 2023

    Accepted: --

    Published Online: Oct. 19, 2023

    The Author Email:

    DOI:10.3788/IRLA20230427

    Topics