To solve the problem of large overshooting and long time needed for stabilization of fast steering mirror (FSM) under traditional proportion-integration-differentiation (PID) control, adaptive fuzzy algorithm was introduced to optimize the control and improve the control performance. Based on the structure and working principle of the voice coil actuator (VCA) fast steering mirror, a closed-loop control model was constructed. The membership function and domain in the controller were optimized, and the adaptive fuzzy incremental PID controller was implemented through field-programmable gate array (FPGA). Simulations and physical tests were conducted. The results show that the adaptive fuzzy incremental PID controller can achieve “zero overshoot” and “zero oscillation” in the overall closed-loop process. At the same time compared with the traditional PID controller, the time required for system stabilization has been shortened by 17.5%, and the system bandwidth has been increased by 16.7%. This method can effectively improve the control effect of the fast steering mirror and can be applied to other systems requiring high-precision control.