As for the image rotation compensation servo mechanism in the aerial imaging system, the disturbances will influence its tracking accuracy.To solve the problem, an improved composite control algorithm is proposed.Firstly, as for the equivalent disturbance, a finite-time hyperbolic sinusoidal tracking differentiator (FSTD) is proposed to construct an improved Nonlinear Disturbance Observer (NDO), which makes the system insensitive to parameter perturbation and more accurate in disturbance estimation.Secondly, in order to improve the tracking accuracy of the system, a New High-order Sliding Mode Controller (N-HSMC) is designed by using the idea of the variable-speed reaching law, which ensures the rapidity of the system and also effectively reduces chattering.Finally, through a comparative simulation, the effectiveness of the proposed method is verified.