When operating mechanical equipment, the number of fault samples marked is small, which leads to low accuracy of the fault diagnosis of the established model. Therefore, this study proposes a defect detection method for eddy current thermal imaging of a workpiece that combines depth learning and domain adaptation. First, the attention mechanism is introduced into the deep residual network ResNet50 to enhance the feature extraction capability of the model. Then, the source and target domain data are sent into the improved ResNet50 network to extract the depth features. The local maximum mean difference is introduced into the full connection layer of the network to reduce the distribution difference between the two domain features to achieve the distribution alignment of related sub-domains. Finally, workpiece metal material defects were detected in the Softmax classifier of the network. The experiment was conducted on the open magnetic tile dataset and eddy current infrared image dataset of the metal plate collected during the experiment. The results show that the method proposed in this paper is highly accurate in detecting and recognizing crack defects in eddy current infrared images. The advantages of the method in this study were verified by visualizing the analysis results using the t-distribution random neighbor embedding method.