Continuous zoom system is a kind of photoelectric imaging device that can switch the field of view, detect and identify the target, continuously. It is fast and stable. Aiming at its control requirements for high accuracy and stability, a design method of fractional-order PID controller based on internal mode control was proposed. The proposed controller has only 3 tuning parameters obtained from desired gain crossover frequency and phase margin. This method can greatly simplifies the design and improves the implementation of controller.The control effect was compared with the traditional integer order PID on Matlab.Simulation results show that the fractional order PID controller increases the steady-state error from 0.1 mm to 0 mm,it has the characteristics of strong anti-interference and robustness. In addition, there is no overshoot and no static error after digital realization. Finally, the digital fractional-order PID was applied to actual continuous zoom system, and the system can obtain clear and stable images, which verifies the effectiveness of the control strategy.