Based on the third-order aberration theory, the aberration of the coaxial four-mirror optical system is analyzed, and the mixed objective function including the primary aberration coefficient of weight distribution and structural constraints is constructed. The improved particle swarm optimization algorithm is used to solve it, and the initial structure with high image quality and specific layout is obtained. Through off-axis processing and the introduction of XY polynomial free-form surface to optimize the initial structure, a free-form surface off-axis four-mirror optical system with focal length of 365 mm, operating band of 0.4?0.7 μm,8?12 μm, F-number of 2, and rectangular field of view of 6°×2° is designed. The results show that the system achieves high energy concentration, the maximum mesh distortion is 2.12%,the optical transfer function exceeding 0.5@50 lp/mm across the entire field of view in the visible band, and the modulation transfer function(MTF) curves in the infrared band almost reach the diffraction limit. The optical system demonstrates excellent imaging quality, when combined with a digital micro-mirror device (DMD), satisfies the dynamic target simulation requirements of the optoelectronic tracking system.