In this paper, we investigate the transmission loss characteristics of gallium nitride (GaN) planar optical waveguides using a finite element simulation model based on the beam propagation method (BPM). To address the high transmission loss in conventional GaN waveguides, we propose process optimization solutions. By constructing a comprehensive transmission loss model, we systematically analyze the impact of waveguide geometric parameters on the transmission characteristics, with a particular focus on investigating the improvement effects of two optimization processes: top etching and back thinning. The experiment results indicate that both processes significantly reduce the waveguide transmission loss, with the top etching process reducing loss from 2.29 dB/mm to 0.19 dB/mm and the back thinning process reducing it to 0.24 dB/mm. Additionally, we analyze the impact of manufacturing defects, such as sidewall angles and surface roughness, on transmission loss. Through parameter optimization, we identify the key dimensions necessary for single mode light transmission. This study provides a theoretical basis and process guidance for the development of low-loss GaN waveguides.