[5] [5] RUFLI M, FERGUSON D, SIEGWART R. Smooth path planning in constrained environments [C]//2009 IEEE International Conference on Robotics and Automation. Kobe: IEEE, 2009: 3780-3785.

[6] [6] FUJIMOTO S, HOOF H, MEGER D. Addressing function approximation error in actor-critic methods [C]//Proceedings of the 35th International Conference on Machine Learning(ICML). Stockholm: PMLR, 2018: 1587-1596.

[7] [7] MNIH V, BADIA A P, MIEZA M, et al. Asynchronous methods for deep reinforcement learning [C]//ICML’16: Proceedings of the 33rd International Conference on International Conference on Machine Learning(ICML). New York: PMLR, 2016: 1928-1937.

[8] [8] GUO M S, WANG Y, LIANG B Y, et al. Robot path planning via deep reinforcement learning with improved reward function [C]//Proceedings of 2021 Chinese Intelligent Systems Conference. Singapore: Springer, 2022: 672-680.

[9] [9] LYU J F, MA X T, YAN J P, et al. Efficient continuous control with double actors and regularized critics [C]//Proceedings of the AAAI Conference on Artificial Intelligence. Pittsburgh: AAAI, 2022: 7655-7663.

[10] [10] HUANG K, LIANG B Y, QUAN J Y, et al. Space manipulator assembly operation technique based on deep residual reinforcement learning [C]//2022 International Conference on Electronics Technology and Artificial Intelligence(ETAI 2022). Chongqing: ETAI, 2022: 012032.

[11] [11] LI Y K, LI D Y, ZHU W S, et al. Constrained motion planning of 7-DOF space manipulator via deep reinforcement learning combined with artificial potential field [J]. Aerospace, 2022, 9(3):163.

[12] [12] HAUSKNECHT M, STONE P. Deep recurrent Q-learning for partially observable mdps [C]//In AAAI Fall Symposium on Sequential Decision Making for Intelligent Agents(AAAI-SDMIA15). Arlington: AIAA, 2015: 29-37.

[13] [13] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Playing atari with deep reinforcement learning [R]. Los Alamos: arXiv Preprint, 2013: arXiv: 1312. 5602.

[14] [14] LAMPLE G, CHAPLOT D S. Playing FPS games with deep reinforcement learning [C]//Proceedings of the AAAI Conference on Artificial Intelligence. San Francisco: AIAA, 2017: 2140-2146.

[15] [15] ANDRYCHOWICZ M, WOLSKI F, RAY A, et al. Hindsight experience replay [R]. Los Alamos: arXiv Preprint, 2017: arXiv: 1707. 01495.

[16] [16] KIM M S, HAN D-K, PARK J-H, et al. Motion planning of robot manipulators for a smoother path using a twin delayed deep deterministic policy gradient with hindsight experience replay [J]. Applied Sciences, 2020, 10(2):575.