An improved disturbance rejection control algorithm was proposed to overcome the velocity fluctuation caused by the disturbance torque in a large ground-based telescope. The algorithm consists of two closed-loop structures, an inner high-bandwidth current loop and an outer speed loop. The inner current loop is a PI controller. The outer loop is a speed controller adopting the linear active disturbance rejection controller. For the speed loop, an extended state observer is used to estimate the system disturbances, and the estimated disturbance is fed into the control system to form a composite correction system. To solve the problem of controller saturation caused by the large dynamic input, an anti-windup control algorithm was induced the input of the extended state observer to guarantee the stability and good dynamic characteristics of the system. The simulation and experimental results show that linear active disturbance rejection controller with anti-windup achieves fast response without overshoot at both high or low speeds as compared with the PI controller. In a low speed smoothing experiment, the algorithm improves the RMS value of speed error from 0.000 68(°)/s to 0.000 32(°)/s. Experimental results demonstrate that the proposed algorithm effectively reduces the velocity fluctuation of the servo system caused by the motor torque ripple and improves its speed smooth.