The differential pressure method, which combines infrared thermal imaging and image processing technologies, is used to detect air infiltration of building exterior windows. Infrared images of the exterior windows of the building were collected using an infrared thermal imager and then processed using infrared image processing technology. Exterior window defects were detected from abnormal areas in the infrared images, and the area of the defects was calculated to establish an infrared detection model for exterior window defects. Based on the indoor and outdoor temperature difference, defect area of the outer window, and air infiltration amount measured in the experiment, a calculation model was established for the amount of air infiltration for the building’s outer window. The model was combined with the infrared detection model for building window defects, to quantitatively analyze the energy consumption caused by the defects. The results show that the maintenance of exterior window defects can reduce energy consumption of the exterior window and improve energy savings. For every 1 cm2 reduction in the air infiltration area of exterior windows, 66146 kJ of energy can be saved annually. For each level of airtightness improvement of exterior windows, 110012 kJ of energy per unit area of exterior windows can be saved annually.