Using multiple UAVs to explore unknown environments can improve the robustness and execution efficiency of exploration tasks.Different from the heuristic method,the multi-agent deep reinforcement learning method eliminates the process of making rules artificially,and takes the UAVs as agents to independently learn more effective “rules” by interacting with the environment.A multi-threaded simulation environment for multiple UAVs is built to provide an environment for cooperative training of multiple UAVs.A Long and Short Term Memory neural network-based shared Multi-Agent Proximal Policy Optimization (LSTM-MAPPO) method is proposed to adapt to the multi-threaded environment,and the global boundary information is added on the basis of the cooperative LSTM-MAPPO method to increase the exploration area of each episode.The numerical experiment results show that:1) Compared with the existing Multi-Agent Depth Deterministic Policy Gradient (MADDPG) method,it can converge stably in later periods of training under the continuous action;2) Compared with the existing LSTM-MAPPO method,its final reward is stably above 5000;and 3) On three different simulation maps,the trained network can realize the stable exploration of more than 70% of the area during the test.