Aiming to solve the problem of difficult quantitative identification of surface defect depth during laser ultrasonic detection, a particle swarm (PSO) optimized quantitative identification method for BP neural network surface rectangular defect depth is proposed. Based on the thermoelastic mechanism, a finite element model for laser ultrasonic detection of aluminium materials containing surface defects was established by using the finite element software COMSOL, the transmission wave signals corresponding to defects of different depths under pulsed laser irradiation were obtained, and then the time domain peak, centre frequency, 3 dB bandwidth in the frequency domain, upper cut-off frequency, and lower cut-off frequency of the transmission wave signals were extracted as the feature vectors of the neural network. A quantitative recognition model of PSO-BP neural network defect depth was developed to achieve the quantitative recognition of defects from 0.1 mm to 3 mm in depth. The calculation results show that the BP neural network optimized by the particle swarm algorithm can accurately identify the depth information of metal surface defects, and the relative error of identification is within 6%, which proves that the neural network model has certain feasibility and accuracy for the identification of rectangular defect depth.