Acta Optica Sinica, Volume. 41, Issue 22, 2215002(2021)

Semantic Visual Odometry Based on Panoramic Annular Imaging

Hao Chen1, Kailun Yang2, Weijian Hu1, Jian Bai1, and Kaiwei Wang1、*
Author Affiliations
  • 1National Engineering Research Center of Optical Instrumentation, Zhejiang University, Hangzhou, Zhejiang 310058, China
  • 2Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Karlsruhe 76131, Germany
  • show less
    References(35)

    [1] Scaramuzza D, Fraundorfer F. Visual odometry tutorial[J]. IEEE Robotics & Automation Magazine, 18, 80-92(2011).

    [2] Engel J, Koltun V, Cremers D. Direct sparse odometry[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 611-625(2018).

    [3] [3] ForsterC, PizzoliM, ScaramuzzaD. SVO: fast semi-direct monocular visual odometry[C]∥2014 IEEE International Conference on Robotics and Automation (ICRA), May 31-June 7, 2014, Hong Kong, China. New York: IEEE Press, 2014: 15- 22.

    [4] Mur-Artal R, Tardós J D. ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 33, 1255-1262(2017).

    [5] Bescos B, Fácil J M, Civera J et al. DynaSLAM: tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 3, 4076-4083(2018).

    [6] Huang Z, Bai J, Hou X Y. Design of combination of panoramic and long focal length optical system with single sensor[J]. Acta Optica Sinica, 33, 0422006(2013).

    [7] Zhang X F, Liu J, Shi Z S et al. Review of deep learning-based semantic segmentation[J]. Laser & Optoelectronics Progress, 56, 150003(2019).

    [8] Lemaire T, Lacroix S. SLAM with panoramic vision[J]. Journal of Field Robotics, 24, 91-111(2007).

    [9] Rituerto A, Puig L, Guerrero J J. Visual SLAM with an omnidirectional camera[C]∥2010 20th International Conference on Pattern Recognition, August 23-26, 2010, Istanbul, Turkey., 348-351(2010).

    [10] Pretto A, Menegatti E, Pagello E. Omnidirectional dense large-scale mapping and navigation based on meaningful triangulation[C]∥2011 IEEE International Conference on Robotics and Automation, May 9-13, 2011, Shanghai, China., 3289-3296(2011).

    [11] Wang Y H, Cai S J, Li S J et al. CubemapSLAM: a piecewise-pinhole monocular fisheye SLAM system[M]. ∥Jawahar C V, Li H D, Mori G, et al. Computer vision-ACCV 2018. Lecture notes in computer science. Cham: Springer, 11366, 34-49(2019).

    [12] Forster C, Zhang Z C, Gassner M et al. SVO: semidirect visual odometry for monocular and multicamera systems[J]. IEEE Transactions on Robotics, 33, 249-265(2017).

    [13] Matsuki H, von Stumberg L, Usenko V et al. Omnidirectional DSO: direct sparse odometry with fisheye cameras[J]. IEEE Robotics and Automation Letters, 3, 3693-3700(2018).

    [14] Campos C, Elvira R. Rodríguez J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 1-17(2021).

    [15] [15] SeokH, LimJ. ROVO: robust omnidirectional visual odometry for wide-baseline wide-FOV camera systems[C]∥2019 International Conference on Robotics and Automation (ICRA), May 20-24, 2019, Montreal, QC, Canada. New York: IEEE Press, 2019: 6344- 6350.

    [16] [16] WonC, SeokH, Cui ZP, et al.OmniSLAM: omnidirectional localization and dense mapping for wide-baseline multi-camera systems[C]∥2020 IEEE International Conference on Robotics and Automation (ICRA), May 31-August 31, 2020, Paris, France. New York: IEEE Press, 2020: 559- 566.

    [17] [17] Lin MJ, Cao QX, Zhang HR. PVO: panoramic visual odometry[C]∥2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM), July 18-20, 2018, Singapore, Singapore. New York: IEEE Press, 2018: 491- 496.

    [18] Ji S P, Qin Z J. Panoramic SLAM for multi-camera rig[J]. Acta Geodaetica et Cartographica Sinica, 48, 1254-1265(2019).

    [19] Liang H J, Sanket N J, Fermüller C et al. SalientDSO: bringing attention to direct sparse odometry[J]. IEEE Transactions on Automation Science and Engineering, 16, 1619-1626(2019).

    [20] Ganti P. SIVO: semantically informed visual odometry and mapping[D]. Canada: University of Waterloo(2018).

    [21] Murali V, Chiu H P, Samarasekera S et al. Utilizing semantic visual landmarks for precise vehicle navigation[C]∥2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), October 16-19, 2017, Yokohama, Japan.(2017).

    [22] Yu C, Liu Z X, Liu X J et al. DS-, 1168-1174(2018).

    [23] Xiao L, Wang J, Qiu X et al. Dynamic-SLAM: semantic monocular visual localization and mapping based on deep learning in dynamic environment[J]. Robotics and Autonomous Systems, 117, 1-16(2019).

    [24] Lu J, Liu Y H, Zhang R F. Semantic-based visual odometry towards dynamic scenes[J]. Laser & Optoelectronics Progress, 58, 061101(2021).

    [25] Bowman S L, Atanasov N, Daniilidis K et al. Probabilistic data association for semantic SLAM[C]∥2017 IEEE International Conference on Robotics and Automation (ICRA), May 29-June 3, 2017, Singapore., 1722-1729(2017).

    [26] Lianos K N, Schönberger J L, Pollefeys M et al. VSO: visual semantic odometry[M]. ∥Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science. Cham: Springer, 11208, 246-263(2018).

    [27] Zou B, Lin S Y, Yin Z S. Semantic mapping based on YOLOv3 and visual SLAM[J]. Laser & Optoelectronics Progress, 57, 201012(2020).

    [28] Scaramuzza D, Martinelli A, Siegwart R. A flexible technique for accurate omnidirectional camera calibration and structure from motion[C]∥Fourth IEEE International Conference on Computer Vision Systems (ICVS'06), January 4-7, 2006, New York, NY, USA., 45(2006).

    [29] Scaramuzza D, Martinelli A, Siegwart R. A toolbox for easily calibrating omnidirectional cameras[C]∥2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 9-15, 2006, Beijing, China., 5695-5701(2006).

    [30] Yang K L, Hu X X, Bergasa L M et al. PASS: panoramic annular semantic segmentation[J]. IEEE Transactions on Intelligent Transportation Systems, 21, 4171-4185(2020).

    [31] Yang K L, Hu X X, Chen H et al. DS-, 457-464(2020).

    [32] Yang K L, Wang K W, Bergasa L M et al. Unifying terrain awareness for the visually impaired through real-time semantic segmentation[J]. Sensors, 18, 1506(2018).

    [34] Bayer J, Faigl J. On autonomous spatial exploration with small hexapod walking robot using tracking camera intel RealSense T265[C]∥2019 European Conference on Mobile Robots (ECMR), September 4-6, 2019, Prague, Czech Republic.(2019).

    [35] Chen H, Wang K, Hu W et al. PALVO: visual odometry based on panoramic annular lens[J]. Optics Express, 27, 24481-24497(2019).

    Tools

    Get Citation

    Copy Citation Text

    Hao Chen, Kailun Yang, Weijian Hu, Jian Bai, Kaiwei Wang. Semantic Visual Odometry Based on Panoramic Annular Imaging[J]. Acta Optica Sinica, 2021, 41(22): 2215002

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Machine Vision

    Received: Mar. 18, 2021

    Accepted: Jun. 3, 2021

    Published Online: Nov. 21, 2021

    The Author Email: Wang Kaiwei (wangkaiwei@zju.edu.cn)

    DOI:10.3788/AOS202141.2215002

    Topics