The plane grating double monochromator is the important part of the instrument for testing diffraction efficiency of plane grating, and in order to accomplish the research of the instrument, the optical system design and simulation of the plane grating double monochromator was investigated. The instrument mainly consists of light source, front monochromator, testing monochromator and detector, and combined with the practical requirement, the corresponding light source, detector and the structure of the optical path were selected. According to the design requirement, the design and simulation of the optical system of the front monochromator and testing monochromator were done respectively, and the spot diagram of the image surface and the results of real ray trace were given. The analysis showed that in the front monochromator the maximum width of the exit spot and the maximum height of the exit spot were 0.33 and 1.83 mm respectively. On the other hand, in the testing monochromator, the maximum width of the exit spot and the maximum height of the exit spot were 0.66 and 7.27 mm, respectively. Through selecting the size of the exit slit properly, the system can work without luminous flux lost, which guarantees the testing precision of the optical system of the instrument.