For high-power laser devices using off-axis multi-pass amplification technology, the technology to suppress self-oscillation is of great research value. It is found in engineering practice that the surface of the structure near the aperture of the main amplification system is relatively bright, thus when the amplifier is working, with sufficient gain, a resonant cavity is formed between the bright structural surface and the cavity mirror to generate self-oscillation and ablation. In this paper, this problem of the device is studied by using Monte Carlo method to simulate the surface of the structure. According to the principle of geometric optics, the reflection model of the laser beam on the surface of the structure is derived. The surface of the main amplification system is simulated using this model, and the parameters of the optical path are used to calculate the relationship between surface roughness and residual reflected light reflected into the amplifier. The surface treatment process is applied to our project. By increasing the surface roughness of the structure, the resonant cavity loss is greater than the gain, which provides a basis for suppressing the self-oscillation of the main amplification system of high-power laser devices.