Review of research on monocular visual servo-based autonomous control of unmanned surface vehicles

Hongkun HE; Ning WANG; Fuyu ZHANG; Bing HAN

doi:10.19693/j.issn.1673-3185.03230

[1] WANG N, GAO Y, YANG C et al. Reinforcement learning-based finite-time tracking control of an unknown unmanned surface vehicle with input constraints[J]. Neurocomputing, 484, 26-37(2022).

[4] WANG N, GAO Y, LIU Y J et al. Self-learning-based optimal tracking control of an unmanned surface vehicle with pose and velocity constraints[J]. International Journal of Robust and Nonlinear Control, 32, 2950-2968(2022).

[6] WANG N, ER M J. Direct adaptive fuzzy tracking control of marine vehicles with fully unknown parametric dynamics and uncertainties[J]. IEEE Transactions on Control Systems Technology, 24, 1845-1852(2016).

[7] WANG N, ER M J. Self-constructing adaptive robust fuzzy neural tracking control of surface vehicles with uncertainties and unknown disturbances[J]. IEEE Transactions on Control Systems Technology, 23, 991-1002(2015).

[8] WANG Y, JIANG B, WU Z G et al. Adaptive sliding mode fault-tolerant fuzzy tracking control with application to unmanned marine vehicles[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51, 6691-6700(2020).

[10] WANG N, KARIMI H R. Successive waypoints tracking of an underactuated surface vehicle[J]. IEEE Transactions on Industrial Informatics, 16, 898-908(2019).

[11] LIU Z X, ZHANG Y M, YUAN C et al. Adaptive path following control of unmanned surface vehicles considering environmental disturbances and system constraints[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51, 339-353(2018).

[12] WANG N, SU S F. Finite-time unknown observer-based interactive trajectory tracking control of asymmetric underactuated surface vehicles[J]. IEEE Transactions on Control Systems Technology, 29, 794-803(2019).

[13] HE S D, WANG M, DAI S L et al. Leader-follower formation control of USVs with prescribed performance and collision avoidance[J]. IEEE Transactions on Industrial Informatics, 15, 572-581(2018).

[18] WANG N, SU S F, YIN J C et al. Global asymptotic model-free trajectory-independent tracking control of an uncertain marine vehicle: an adaptive universe-based fuzzy control approach[J]. IEEE Transactions on Fuzzy Systems, 26, 1613-1625(2018).

[20] SHIRAI Y, INOUE H. Guiding a robot by visual feedback in assembling tasks[J]. Pattern Recognition, 5, 99-108(1973).

[21] [21] HILL J, PARK W T. Real time control of a robot with a mobile camera[C]Proceedings of the 9th International Symposium on Industrial Robots. Washington, DC, USA: SME, 1979: 233−246.

[24] HORAUD R, DORNAILKA F, ESPIAU B. Visually guided object grasping[J]. IEEE Transactions on Robotics and Automation, 14, 525-532(1998).

[25] GUENARD N, HAMEL T, MAHONY R. A practical visual servo control for an unmanned aerial vehicle[J]. IEEE Transactions on Robotics, 24, 331-340(2008).

[26] CHEN J, DIXON W E, DAWSON M et al. Homography-based visual servo tracking control of a wheeled mobile robot[J]. IEEE Transactions on Robotics, 22, 406-415(2006).

[27] KRUPÍNSKI S, ALLIBERT G, HUA M D et al. An inertial-aided homography-based visual servo control approach for (almost) fully actuated autonomous underwater vehicles[J]. IEEE Transactions on Robotics, 33, 1041-1060(2017).

[28] HUTCHINSON S, HAGER G D, CORKE P I. A tutorial on visual servo control[J]. IEEE Transactions on Robotics and Automation, 12, 651-670(1996).

[29] CHAUMETTE F, HUTCHINSON S. Visual servo control. part I: basic approaches[J]. IEEE Robotics & Auto-mation Magazine, 13, 82-90(2006).

[30] CHAUMETTE F, HUTCHINSON S. Visual servo control. part II: advanced approaches[J]. IEEE Robotics & Automation Magazine, 14, 109-118(2007).

[31] STANIAK M, ZIELIŃSKI C. Structures of visual servos[J]. Robotics and Autonomous Systems, 58, 940-954(2010).

[32] AZIZIAN M, KHOSHNAM M, NAJMAEI N et al. Visual servoing in medical robotics: a survey. part I: endo-scopic and direct vision imaging-techniques and applications[J]. The International Journal of Medical Robotics and Computer Assisted Surgery, 10, 263-274(2014).

[33] AZIZIAN M, NAJMAEI N, KHOSHNAM M et al. Visual servoing in medical robotics: a survey. part II: tomographic imaging modalities-techniques and applications[J]. The International Journal of Medical Robotics and Computer Assisted Surgery, 11, 67-79(2015).

[40] WU J H, JIN Z H, LIU A D et al. A survey of learning-based control of robotic visual servoing systems[J]. Journal of the Franklin Institute, 359, 556-577(2021).

[41] HUANG Y, SU J B. Visual servoing of nonholonomic mobile robots: a review and a novel perspective[J]. IEEE Access, 7, 134968-134977(2019).

[42] SØRENSEN A J. A survey of dynamic positioning control systems[J]. Annual Reviews in Control, 35, 123-136(2011).

[43] [43] WANG J, LIU J Y, YI H. A modular designed dynamic positioning experiment system its application on a ship model[C]OCEANS 2016. Shanghai, China: IEEE, 2016: 1−6.

[44] HU B B, Zhang H T. Bearing-only motional target-surrounding control for multiple unmanned surface vessels[J]. IEEE Transactions on Industrial Electronics, 69, 3988-3997(2022).

[45] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 60, 91-110(2004).

[46] [46] BAY H, TUYTELAARS T, GOOL L V. SURF: speeded up robust features[C]Proceedings of the 9th European conference on Computer Vision(ECCV''06 ) Volume Part I. Berlin, Germany: Springer, 2006: 404−417.

[47] [47] LEUTENEGGER S, CHLI M, SIEGWART R Y. BRISK: Binary robust invariant scalable keypoints[C]Proceedings of International Conference on Computer Vision. Barcelona, Spain: IEEE, 2011: 2548−2555.

[48] TSAI R. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses[J]. IEEE Journal on Robotics and Automation, 3, 323-344(1987).

[50] ZHANG Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1330-1334(2000).

[51] [51] FOSSEN T I. Hbook of marine craft hydrodynamics motion control[M]. New Yk: John Wiley & Sons, 2011.

[52] DO K D, PAN J. Global robust adaptive path following of underactuated ships[J]. Automatica, 42, 1713-1722(2006).

[53] DAI S L, HE S D, Lin H et al. Platoon formation control with prescribed performance guarantees for USVs[J]. IEEE Transactions on Industrial Electronics, 65, 4237-4246(2017).

[54] ZHAO J X, WANG J H, ZHENG X. Visual servo-based method for bridge crossing of unmanned surface vehicle[J]. International Core Journal of Engineering, 6, 276-284(2020).

[55] [55] ZHAO J X, WANG J H, ZHENG X, et al. Research on USV sailing through the bridge with visual servo[C]Global Oceans 2020: Singape U. S. Gulf Coast. Singape: IEEE, 2020: 1−6.

[56] [56] EKELMANN J, BUTKA B. Seagle 3.0 an autonomous surface vehicle[C]Proceedings of IEEE Southeastcon. lo, FL, USA: IEEE, 2012: 1−5.

[58] [58] MARTINS A, ALMEIDA J M, FERREIRA H, et al. Autonomous surface vehicle docking manoeuvre with visual infmation[C]Proceedings of IEEE International Conference on Robotics Automation. Rome, Italy: IEEE, 2007: 4994−4999.

[59] [59] DUNBABIN M, LANG B, WOOD B. Visionbased docking using an autonomous surface vehicle[C]Proceedings of IEEE International Conference on Robotics Automation. Pasadena, CA,USA: IEEE, 2008: 26−32.

[60] KIM Y H, LEE S W, Yang H S et al. Toward autonomous robotic containment booms: visual servoing for robust inter-vehicle docking of surface vehicles[J]. Intelligent Service Robotics, 5, 1-18(2012).

[62] WANG N, HE H K. Adaptive homography-based visual servo for micro unmanned surface vehicles[J]. The International Journal of Advanced Manufacturing Technology, 105, 4875-4882(2019).

[63] WANG N, HE H K. Extreme learning-based monocular visual servo of an unmanned surface vessel[J]. IEEE Transactions on Industrial Informatics, 17, 5152-5163(2021).

[64] WANG N, HE H K. Dynamics-level finite-time fuzzy monocular visual servo of an unmanned surface vehicle[J]. IEEE Transactions on Industrial Electronics, 67, 9648-9658(2020).

[66] [66] HE H K, WANG N. Monocular visual servo of unmanned surface vehicles with viewfield constraints[C]Proceedings of the 33rd Chinese Control Decision Conference. Kunming, China: IEEE, 2021: 973−978.

[67] MARCHAND E, UCHIYAMA H, SPINDLER F. Pose estimation for augmented reality: a hands-on survey[J]. IEEE Transactions on Visualization and Computer Graphics, 22, 2633-2651(2015).

[68] WANG K, LIU Y H, LI L Y. Vision-based tracking control of underactuated water surface robots without direct position measurement[J]. IEEE Transactions on Control Systems Technology, 23, 2391-2399(2015).

[69] LEFEBER E, PETTERSEN K Y, NIJMEIJER H. Track-ing control of an underactuated ship[J]. IEEE Trans-actions on Control Systems Technology, 11, 52-61(2003).

[70] [70] WANG K, LIU Y H, LI L Y. Visionbased tracking control of nonholonomic mobile robots without position measurement[C]Proceedings of 2013 IEEE International Conference on Robotics Automation. Karlsruhe, Germany: IEEE, 2013: 5265−5270.

[71] WANG K, LIU Y H, LI L Y. Visual servoing trajectory tracking of nonholonomic mobile robots without direct position measurement[J]. IEEE Transactions on Robotics, 30, 1026-1035(2014).

[72] LIANG X W, LIU Y H, WAMG H S et al. Leader-following formation tracking control of mobile robots without direct position measurements[J]. IEEE Transactions on Automatic Control, 61, 4131-4137(2016).

[73] WANG K, LIU Y H, LI L Y. A simple and parallel algorithm for real-time robot localization by fusing monocular vision and odometry/AHRS sensors[J]. IEEE/ASME Transactions on Mechatronics, 19, 1447-1457(2014).

[74] WANG N, LV S L, ZHANG W D et al. Finite-time observer based accurate tracking control of a marine vehicle with complex unknowns[J]. Ocean Engineering, 145, 406-415(2017).

[75] WANG N, LV S L, ER M J et al. Fast and accurate trajectory tracking control of an autonomous surface vehicle with unmodeled dynamics and disturbances[J]. IEEE Transactions on Intelligent Vehicles, 1, 230-243(2017).

[76] WANG N, KARIMI H R, LI H Y et al. Accurate trajectory tracking of disturbed surface vehicles: a finite-time control approach[J]. IEEE/ASME Transactions on Mechatronics, 24, 1064-1074(2019).

[77] ZHENG D L, WANG H S, CHEN W D et al. Planning and tracking in image space for image-based visual servoing of a quadrotor[J]. IEEE Transactions on Industrial Electronics, 65, 3376-3385(2018).

[78] ZHANG X T, FANG Y C, ZHANG X B et al. Attitude-constrained time-optimal trajectory planning for rotorcrafts: theory and application to visual servoing[J]. IEEE/ASME Transactions on Mechatronics, 25, 1912-1921(2020).

[79] WANG R H, ZHANG X B, FANG Y C et al. Virtual-goal-guided RRT for visual servoing of mobile robots with FOV constraint[J]. IEEE Transactions on Systems, Man, and Cybernetics:Systems, 52, 2073-2083(2021).

[80] CHESI G, HASHIMOTO K, PRATTICHIZZO D et al. Keeping features in the field of view in eye-in-hand visual servoing: a switching approach[J]. IEEE Transactions on Robotics, 20, 908-914(2004).

[81] MURRIERI P, FONTANELLI D, BICCHI A. A hybrid-control approach to the parking problem of a wheeled vehicle using limited view-angle visual feedback[J]. The International Journal of Robotics Research, 23, 437-448(2004).

[82] GANS N R, HUTCHINSON S A. Stable visual servoing through hybrid switched-system control[J]. IEEE Transactions on Robotics, 23, 530-540(2007).

[83] BECHLIOULIS C P, HESHMATI-ALAMDARI S, KARRAS G C et al. Robust image-based visual servoing with prescribed performance under field of view constraints[J]. IEEE Transactions on Robotics, 35, 1063-1070(2019).

[84] MIAO Z, ZHONG H, LIN J et al. Vision-based formation control of mobile robots with FOV constraints and unknown feature depth[J]. IEEE Transactions on Control Systems Technology, 29, 2231-2238(2020).

[85] LIN J, MIAO Z Q, ZHONG H et al. Adaptive image-based leader-follower formation control of mobile robots with visibility constraints[J]. IEEE Transactions on Industrial Electronics, 68, 6010-6019(2020).

[86] FANG Y C, LIU X, ZHAGN X B. Adaptive active visual servoing of nonholonomic mobile robots[J]. IEEE Transactions on Industrial Electronics, 59, 486-497(2011).

[87] CHEN X H, JIA Y M. Adaptive leader-follower formation control of non-holonomic mobile robots using active vision[J]. IET Control Theory & Applications, 9, 1302-1311(2015).

[88] WAI R J, LIN Y W. Adaptive moving-target tracking control of a vision-based mobile robot via a dynamic petri recurrent fuzzy neural network[J]. IEEE Transactions on Fuzzy Systems, 21, 688-701(2012).

[89] FREDA L, ORIOLO G. Vision-based interception of a moving target with a nonholonomic mobile robot[J]. Robotics and Autonomous Systems, 55, 419-432(2007).

[90] MA H X, ZOU W, ZHU Z et al. Moving to OOP: an active observation approach for a novel composite visual servoing configuration[J]. IEEE Transactions on Instrumentation and Measurement, 70, 1-15(2020).

[94] WANG N, LI H. Leader-follower formation control of surface vehicles: a fixed-time control approach[J]. ISA Transactions, 124, 356-364(2022).

[95] KANJANAWANISHKUL K. Coordinated path following for mobile robots using a virtual structure strategy with model predictive control[J]. Automatika, 55, 287-298(2014).

[96] BALCH T, ARKIN R C. Behavior-based formation control for multirobot teams[J]. IEEE Transactions on Robotics and Automation, 14, 926-939(1998).

[97] [97] WANG J, LIU J Y, YI H. Fmation control of unmanned surface vehicles with vision sens constraints[C]MTSIEEE Oceans, Washington, DC, USA: IEEE, 2015: 1−8.

[98] [98] PANAGOU D, KYRIAKOPOULOS K J. Cooperative fmation control of underactuated marine vehicles f target surveillance under sensing communication constraints[C]Proceedings of 2013 IEEE International Conference on Robotics Automation. Karlsruhe, Germany: IEEE, 2013: 1871−1876.

[99] [99] WANG J, LIU J Y, YI H. Fmation control of unmanned surface vehicles with sensing constraints using exponential remapping method[J]. Mathematical Problems in Engineering, 2017,2027(pt.8): 1−14.

[100] DAI S L, HE S D, CHEN X et al. Adaptive leader-follower formation control of nonholonomic mobile robots with prescribed transient and steady-state performance[J]. IEEE Transactions on Industrial Informatics, 16, 3662-3671(2020).

[101] [101] REYNOLDS C W. Flocks, herds schools: a distributed behavial model[J]. ACM SIGGRAPH Computer Graphics, 1987, 21（4）: 25–34.

[102] VISCEK T, CZIRÓK A, BEN-JACOB E et al. Novel type of phase transition in a system of self-driven particles[J]. Physical Review Letters, 75, 1226-1229(1995).

[103] CUCKER F, SMALE S. Emergent behavior in flocks[J]. IEEE Transactions on Automatic Control, 52, 852-862(2007).

[104] COUZIN I D, KRAUSE J, JAMES R et al. Collective memory and spatial sorting in animal groups[J]. Journal of Theoretical Biology, 218, 1-11(2002).

[105] SU H S, WANG X F, LIN Z L. Flocking of multi-agents with a virtual leader[J]. IEEE Transactions on Automatic Control, 54, 293-307(2009).

[106] CHEAH C C, HOU S P, SLOTINE J J E. Region-based shape control for a swarm of robots[J]. Automatica, 45, 2406-2411(2009).

[107] QIN Z H, LIN Z, YANG D M et al. A task-based hierarchical control strategy for autonomous motion of an unmanned surface vehicle swarm[J]. Applied Ocean Research, 65, 251-261(2017).

[108] LIANG X, QU X R, WANG N et al. A novel distributed and self-organized swarm control framework for underactuated unmanned marine vehicles[J]. IEEE Access, 7, 112703-112712(2019).

[109] LIANG X, QU X R, WANG N et al. Swarm control with collision avoidance for multiple underactuated surface vehicles[J]. Ocean Engineering, 191, 106-116(2019).

[110] TAN G G, ZOU J, ZHUANG J Y et al. Fast marching square method based intelligent navigation of the unmanned surface vehicle swarm in restricted waters[J]. Applied Ocean Research, 95, 102-117(2020).

[111] TAN G G, ZHUANG J Y, ZOU J et al. Artificial potential field-based swarm finding of the unmanned surface vehicles in the dynamic ocean environment[J]. International Journal of Advanced Robotic Systems, 17, 1-16(2020).

[112] ZHUANG J Y, ZHANG L, QIN Z H et al. Motion control and collision avoidance algorithms for unmanned surface vehicle swarm in practical maritime environment[J]. Polish Maritime Research, 26, 107-116(2019).

[113] OLFATI-SABER R. Flocking for multi-agent dynamic systems: algorithms and theory[J]. IEEE Transactions on Automatic Control, 51, 401-420(2006).

[114] FRANCHI A, SECCHI C, SON H I et al. Bilateral teleoperation of groups of mobile robots with time-varying topology[J]. IEEE Transactions on Robotics, 28, 1019-1033(2012).