In order to meet the aerodynamic requirements, the dome or optical window with conformal thin-walled structure has become the development trend of high-speed aircraft in the future. However, the cutting force will change with the axial position in the machining process of such parts, and one-time machining is difficult to meet the accuracy requirements. It is necessary to control the surface error caused by the change of cutting force by on-machine measurement and compensating machining. Taking the ultra-precision lathe as the motion platform, the on-machine measurement system was designed for thin-walled optical parts with high steepness, the optimization algorithm was studied for the distribution of measuring points, and the measuring efficiency and accuracy were realized at the same time. The correction model for thermal deformation error was established to improve the accuracy of on-machine measurement for thin-walled optical parts with high steepness. The on-machine measurement was used to provide guidance for compensating machining of a thin-walled dome, the surface error was reduced from PV (peak to valley) 3.1 μm to PV 0.7 μm, the coaxiality is controlled to 1.02 μm, and the performance requirements are satisfied for optical system.