[4] [4] WANG D W,FAN T X,HAN T,et al.A two-stage reinf-orcement learning approach for multi-UAV collision avoi- danceunder imperfect sensing［J］.IEEE Robotics and Automation Letters,2020,5(2):3098-3105.

[6] [6] CHENG Y,SONG Y.Autonomous decision-making generation of UAV based on soft actor-critic algorithm［C］//The 39th Chinese Control Conference (CCC).Shenyang:IEEE,2020.doi:10.23919/CCC50068.2020.9188886.

[7] [7] SCHULMAN J,WOLSKI F,DHARIWAL P,et al.Proximal policy optimization algorithms［EB/OL］.(2017-07-01)［2022-08-23］.https://ui.adsabs.harvard.edu/abs/2017a rXiv170706347S/abstract.

[8] [8] ESCHMANN J.Reward function design in reinforcement learning［J］.Reinforcement Learning Algorithms:Analysis and Applications,2021,883:25-33.

[11] [11] PROENA H,NEVES J C.Deep-PRWIS:periocular recognition without the IRIS and sclera using deep learning frameworks［J］.IEEE Transactions on Information Forensics and Security,2018,13(4):888-896.

[12] [12] NANDY A,BISWAS M.Reinforcement learning:with open AI,tensortlow and keras using python［M］.Berkeley:Apress,2017.

[13] [13] VZQUEZ-CANTELI J R,KMPF J,HENZE G,et al.CityLearn v1.0:an OpenAI Gym environment for demand response with deep reinforcement learning［C］//Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings,Cities,and Transportation.New York,NY:BuildSys,2019:356-357.

[15] [15] SCHULMAN J,MORITZ P,LEVINE S,et al.High-dimensional continuous control using generalized advantage estimation［EB/OL］.(2015-06-08)［2022-08-23］.https://arxiv.org/abs/1506.02438.

[17] [17] HAARNOJA T,ZHOU A,ABBEEL P,et al.Soft actor-critic:off-policy maximum entropy deep reinforcement learning with a stochastic actor［C］//Proceedings of the 35th International Conference on Machine Learning.Stockholm:PMLR,2018:1861-1870.