To overcome the shortcomings of off-axis Three Mirror Anastigmatic (TMA) optical systems in larger sagittally FOVs and smaller meridionally FOVs, this paper designs a bidirectional TMA optical system for cryogenic infrared target simulation system based on the symmetric rule of optical system. Both the sagittally and meridionally FOVs are 5°,the larger FOV optical system at 3-5 μm has a focal length of 400 mm and a F number of 8. By using the structural parameters of the optical system and the non-spherical coefficients of the mirrors to adjust and correct the eccentric and tilt of the three-mirror and to eliminate distortion and other aberrations, the system shows that the MTF is better than 0.71 at 6.5 lp/mm, the aberration of the whole FOV is λ/250(RMS) and the maximum radius of diffuse point is less than 7.0 μm(RMS), which achieves diffraction limit. Moreover, the maximum distortion in the whole FOV is less than 0.04%. Finally, it proves that the optical system has good imaging quality both in the visible and infrared spectral regions.