Aiming at the multi-objective optimization problem in reconfigurable intelligence surface (RIS)-assisted integrated aerial-terrestrial networks (IATNs), an algorithmic framework is proposed to jointly optimize the active transmit beamforming matrix, passive RIS, reflect beamforming matrix and unmanned aerial vehicle (UAV) trajectory using deep reinforcement learning (DRL). An algorithmic framework using DRL is proposed to jointly optimize the active transmit beamforming matrix, passive RIS, and UAV trajectory. A multi-objective constrained optimization model for system and rate maximization is established using the base station active beamforming technique and non-orthogonal multiple access (NOMA) technique. The DRL-based deep deterministic policy gradient (DDPG) framework is used to optimize the base station active transmit beamforming matrix, RIS passive reflective beamforming matrix and UAV trajectory. The results show that the DDPG framework integrating adaptive operator mechanism outperforms the traditional iterative optimization standard scheme in terms of system performance, execution time, and higher computational speed, and the system and rate can be improved by about 18%.