Photovoltaic fault detection is of great significance to the intelligent operation and maintenance of photovoltaic power plants. To address the problem of small targets and difficult detection of hot spots in infrared images of photovoltaic modules, an ran infrared hot spot fault detection model for PV modules based on improved Faster R-CNN is studied. Swin Transformer is employed as the feature extraction module in the Faster R-CNN model to capture the global information from the images and establish dependencies between the features, thereby enhancing the modeling capability of the model. Furthermore, the BiFPN is utilized for feature fusion, improving the issue of thermal spot faults that are easily ignored by the model due to the small target and inconspicuous features. Additionally, to suppress interference from background and noise in photovoltaic infrared images, a lightweight attention module called CBAM is incorporated to enable the model to focus more on important channels and key regions, so as to improve the accuracy of thermal spot fault detection. Experimental evaluations are conducted on a self-built dataset of photovoltaic component images, resulting in an impressive detection accuracy of 91.5%, which validates the effectiveness of the proposed model for detecting thermal spot faults in photovoltaic components.