Determining the types and sizes of surface defects is crucial for an evaluation of the surface quality of precision optical components. We propose a cascaded inversion algorithm based on a decision tree model to address the limitations of traditional inversion algorithms in terms of inversion dimension and scale when angle-resolved scattering signals are used to invert the structural characteristic parameters of surface defects. To construct the dataset needed to train the model, an electromagnetic simulation of the angle-resolved scattering system was established using the finite difference time domain method, and the dataset was obtained through simulation calculations. The inversion results for the test set data show that the proposed algorithm is able to predict the defect type and depth with a precision having an area under the curve of 0.99 and an average R² of 0.932, expanding the inversion dimension. The algorithm also accurately predicts the width of defects at different defect depths with an average R² of 0.997, increasing the scale of inversion. The proposed algorithm offers a new approach to the precise quantitative analysis of small defects on the surface of optical components.