Fast steering mirror (FSM) driven by piezoelectric ceramics has been widely used in the execution of adaptive optical systems. In order to accurately model its hysteresis effect, this paper proposes an IDE-BPNN modeling method for FSM. Based on Madelung’s rule, a symmetric hysteresis operator is constructed by least squares method as the basic description of hysteresis motion, and the training dataset is extended. The improved differential evolution algorithm (IDE) is used to train BP neural network (BPNN). The experiment shows that when a 30 Hz attenuated sine signal is input, the single-axis maximum error of the IDE-BPNN model is 0.745 μrad, the normalized maximum error is 0.87%, and the normalized root mean square error is 0.36%. Compared with the least squares model, the error of this model is greatly reduced and has good practical value.