According to the vacuum and cryogenic test requirements of remote sensors, a set of infrared target background simulators that could work stably in low temperature and vacuum environments was designed and built. The simulator is mainly composed of cold diaphragm, vacuum and low temperature surface source blackbody and three-dimensional electric moving table. The cold diaphragm simulates the detection background, and the micro holes distributed on the cold diaphragm are used to simulate the detection point targets. By effectively controlling the thermal insulation and temperature control between the target simulator and the background simulator and the thermal insulation between the background simulator and the remote sensor to be tested, the stable test is realized. In addition, combining simulation optimization with practical experience, the influence of the thickness of the cold stop plate, the target phase and the collimator of the simulator is removed through simulation calculation, which effectively reduces the measurement uncertainty of the system. The simulation analysis method and the verification results in this paper have reference significance for the detection experiment of signal-to-noise ratio of infrared remote sensor point target detection.