Opto-Electronic Engineering, Volume. 51, Issue 10, 240180(2024)

A 3D reconstruction method based on multi-view fusion and hand-eye coordination for objects beyond the visual field

Botao Zhang1,3、*, Zhengqiang Li1, Chaohao Hua3, Jialong Xie2, and Qiang Lu1,3
Author Affiliations
  • 1School of Automation,Hangzhou Dianzi University,Hangzhou,Zhejiang 310018,China
  • 2School of Control Science and Engineering,Shandong University,Jinan,Shandong 250061,China
  • 3International Joint Laboratory of Autonomous Robotic Systems,Hangzhou Dianzi University,Hangzhou,Zhejiang 310018,China
  • show less
    References(33)

    [1] Yu X Y, Cheng Z Y, Zhang Y K et al. Point cloud modeling and slicing algorithm for trajectory planning of spray painting robot[J]. Robotica, 39, 2246-2267(2021).

    [2] Ge J M, Deng Z H, Li Z Y et al. Adaptive parameter optimization approach for robotic grinding of weld seam based on laser vision sensor[J]. Rob Comput Integr Manuf, 82, 102540(2023).

    [3] Nigro M, Sileo M, Pierri F et al. Assembly task execution using visual 3D surface reconstruction: an integrated approach to parts mating[J]. Rob Comput Integr Manuf, 81, 102519(2023).

    [4] Yang L, Liu Y H, Peng J Z et al. A novel system for off-line 3D seam extraction and path planning based on point cloud segmentation for arc welding robot[J]. Rob Comput Integr Manuf, 64, 101929(2020).

    [5] Lee I D, Seo J H, Kim Y M et al. Automatic pose generation for robotic 3-D scanning of mechanical parts[J]. IEEE Trans Rob, 36, 1219-1238(2020).

    [6] Qian J M, Feng S J, Xu M Z et al. High-resolution real-time 360° 3D surface defect inspection with fringe projection profilometry[J]. Opt Lasers Eng, 137, 106382(2021).

    [7] Xu Z, Kang R, Lu R D. 3D reconstruction and measurement of surface defects in prefabricated elements using point clouds[J]. J Comput Civil Eng, 34, 04020033(2020).

    [8] Zhang S. High-speed 3D shape measurement with structured light methods: a review[J]. Opt Lasers Eng, 106, 119-131(2018).

    [9] Zuo C, Feng S J, Huang L et al. Phase shifting algorithms for fringe projection profilometry: a review[J]. Opt Lasers Eng, 109, 23-59(2018).

    [10] Xiang Z L, Zhang Q C, Wu Z J. 3D shape measurement and texture mapping method based on structured light projection[J]. Opto-Electron Eng, 49, 220169(2022).

    [11] Lyu C, Bai Y, Yang J et al. An iterative high dynamic range image processing approach adapted to overexposure 3D scene[J]. Opt Lasers Eng, 124, 105831(2020).

    [12] Yang S, Li B C, Cao Y P et al. Noise-resilient reconstruction of panoramas and 3D scenes using robot-mounted unsynchronized commodity RGB-D cameras[J]. ACM Trans Graphics, 39, 152(2020).

    [13] Ge J H, Li J X, Peng Y P et al. Online 3-D modeling of complex workpieces for the robotic spray painting with low-cost RGB-D cameras[J]. IEEE Trans Instrum Meas, 70, 5011013(2021).

    [14] Li X, Liu F, Shao X P. Research progress on polarization 3D imaging technology[J]. J Infrared Millim Waves, 40, 248-262(2021).

    [16] Zhu Z M, Zeng X N, Long W Q et al. Three-dimensional reconstruction of polarized ambient light separation in complex illumination[J]. Opt Express, 32, 13932-13945(2024).

    [18] Chu D Y, Zhang G H, Song R J et al. Priori knowledge assisted dynamic 3D shape measurement with fringe projection[J]. Opto-Electron Eng, 49, 210449(2022).

    [19] Li C D, Yu L, Fei S M. Large-scale,real-time 3D scene reconstruction using visual and IMU sensors[J]. IEEE Sens J, 20, 5597-5605(2020).

    [20] Ding L J, Dai S G, Mu P G. CAD-based path planning for 3D laser scanning of complex surface[J]. Procedia Comput Sci, 92, 526-535(2016).

    [21] Koutecký T, Paloušek D, Brandejs J. Sensor planning system for fringe projection scanning of sheet metal parts[J]. Measurement, 94, 60-70(2016).

    [22] Malhan R, Jomy Joseph R, Bhatt P M et al. Algorithms for improving speed and accuracy of automated three-dimensional reconstruction with a depth camera mounted on an industrial robot[J]. J Comput Inf Sci Eng, 22, 031012(2022).

    [24] Zong Y L, Liang J, Pai W Y et al. A high-efficiency and high-precision automatic 3D scanning system for industrial parts based on a scanning path planning algorithm[J]. Opt Lasers Eng, 158, 107176(2022).

    [25] Gao J, Li F, Zhang C et al. A method of D-type weld seam extraction based on point clouds[J]. IEEE Access, 9, 65401-65410(2021).

    [27] Chen P, Ren J J, Wang H X et al. Equal-scale structure from motion method based on deep learning[J]. Opto-Electron Eng, 46, 190006(2019).

    [28] Wang J S, Tao B, Gong Z Y et al. A mobile robotic measurement system for large-scale complex components based on optical scanning and visual tracking[J]. Rob Comput Integr Manuf, 67, 102010(2021).

    [29] Wang J S, Gong Z Y, Tao B et al. A 3-D reconstruction method for large freeform surfaces based on mobile robotic measurement and global optimization[J]. IEEE Trans Instrum Meas, 71, 5006809(2022).

    [30] Tao S J, Bai R L. Point cloud registration method based on key point optimization after downsampling[J]. Appl Res Comput, 38, 904-907(2021).

    [32] Tsai R Y, Lenz R K. A new technique for fully autonomous and efficient 3D robotics hand/eye calibration[J]. IEEE Trans Rob Autom, 5, 345-358(1989).

    Tools

    Get Citation

    Copy Citation Text

    Botao Zhang, Zhengqiang Li, Chaohao Hua, Jialong Xie, Qiang Lu. A 3D reconstruction method based on multi-view fusion and hand-eye coordination for objects beyond the visual field[J]. Opto-Electronic Engineering, 2024, 51(10): 240180

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Article

    Received: Aug. 4, 2024

    Accepted: Sep. 26, 2024

    Published Online: Jan. 2, 2025

    The Author Email: Botao Zhang (张波涛)

    DOI:10.12086/oee.2024.240180

    Topics