When using ground-based medium wave infrared measurement system, gray value drift is one of the significant errors in radiation calibration and measurement. By investigating the dependence between the ambient temperature with the output gray value of infrared system, the law of gray drift with ambient temperature was summarized, the reason of gray drift was found, the relationship between ambient temperature and gray drift was deduced, and a method based on ambient temperature was proposed to compensate the gray value drift. The results indicated that the method in this paper can compensate gray value drift effectively and reduce the gray value drift caused by ambient temperature.