Line-structured light scanning measurement technology based on galvanometer mirrors has wide applications due to its outstanding accuracy and high efficiency in the field of 3D measurement. However, the difficulty in precisely aligning the laser with the galvanometer's rotation axis during installation significantly impacts the calibration accuracy of the galvanometer coordinate axis, thereby affecting the overall measurement system's accuracy. Consequently, the precise calibration of the scanning galvanometer coordinate axis has become one of the key challenges in achieving high-precision measurements. A nonlinear algorithm is proposed which uses iteratively approximating to find the optimal solution based on the nonlinear characteristics of the equations to be solved, and the precision calibration of the galvanometer coordinate axis is realized. The theory and processing steps of this calibration method are introduced, and by conducting multiple measurements on a triangular calibration block, it is found that the absolute error of this method ranges from 0.201 1 mm to 0.492 7 mm, lower than the values of 0.313 5 mm to 0.770 3 mm using the traditional least squares method, which indicates that the proposed method significantly improves both accuracy and stability. Therefore, the method provides a new solution for high-precision measurement based on scanning galvanometers, and presents substantial application values.