An image processing method based on the HSV space model with an improved K-means clustering algorithm is proposed to accurately identify and extract the hot spot part of photovoltaic modules. First, the infrared image is transformed into the HSV space and bilaterally filtered to remove noise and improve the image contrast. Second, the Gaussian kernel function is used to extract the image grayscale probability density function, and then the initial clustering center is obtained. Finally, K-means clustering is applied to the image using prior knowledge to extract and quantify the hot spot defects. The research results show that the method can quickly detect and locate the hotspot position and calculate the degree of damage to the photovoltaic panel, and has high accuracy, good sensitivity, and stability.