In the radiometric calibration of a large military infrared imaging simulator when the field stop changes, the infrared radiometer needs to be used under the measurement mode of the underfilled detector. Yet, existing studies have neglected the adverse influences of the image inconsistency between calibration and measurement under this mode on the measured results that lead to large errors in radiometric calibration of large military infrared imaging simulators when the field stop changes. In response to the above problem, theoretical research is conducted on the effect of detector surface heterogeneity on the response voltage in different imaging states. Then, a radiometric calibration method based on the image consistency of calibration and measurement is proposed to reduce the measurement error caused by image inconsistency. Finally, experiments are carried out on calibration of the infrared radiometer and radiation measurement of a certain type of infrared imaging simulator in different imaging states. Experimental results show that if the imaging states of calibration and measurement are not consistent, a large measurement error will be produced. The measurement error introduced by the image inconsistency between calibration and measurement when the field stop changes can be effectively avoided by the proposed radiometric calibration method based on the image consistency of calibration and measurement, which ensures the effectiveness of radiometric calibration of large military infrared imaging simulators.