The primary objective of our study is to enhance the quality and automation of keyhole Tungsten inert gas (TIG) welding by developing a real-time visual monitoring system using deep learning techniques, specifically convolutional neural networks (CNNs). Keyhole TIG welding is a widely recognized advanced welding method known for its efficiency and precision. Despite its advantages, traditional methods of monitoring and quality control are limited by their reliance on manual feature selection and subjective judgment, resulting in inconsistent results and increased labor costs. We aim to deal with these limitations by leveraging the capabilities of deep learning to automatically and accurately identify various welding states, thus improving the overall welding process. The importance and necessity of our study are underscored by the growing demand for high-quality welds in various industries, such as aerospace, automotive, and construction fields, where precision and reliability are paramount. Traditional welding quality control methods often fall short in meeting these rigorous standards due to their dependence on human operators, who may have differences in skill and consistency. By developing an automated system that employs state-of-the-art deep learning algorithms, we aim to provide a more reliable and efficient solution, ultimately leading to improved production outcomes and reduced operational costs.

The experimental setup integrates a specialized welding camera into a robotic system equipped with welding torches, deployed at a 45° angle to the welding path to comprehensively capture the weld pool and the area near the welding arc (Fig. 1). The camera system is designed to filter out the intense arc light and enhance the details within the weld pool. Meanwhile, the experiments are conducted under a range of parameters such as gas flow rate, traveling speed, voltage, and currents to simulate different welding conditions and induce various types of defects like burn-through, contamination, lack of fusion, misalignment, and incomplete penetration (Table 1). Our study employs the ResNet-18 deep learning architecture, chosen for its effectiveness in mitigating gradient vanishing problems during the training of deep neural networks. The ResNet-18 architecture’s advantage lies in its residual learning framework, which allows for the training of much deeper networks without degradation problems. This characteristic is crucial for accurately identifying subtle differences in welding states from visual data. Data augmentation techniques such as random rotation, horizontal flipping, and brightness and contrast adjustments are applied to enhance the diversity of the training dataset. These techniques help prevent overfitting by ensuring that the model is exposed to a wide variety of data during training. The model’s training process is optimized by adopting the Adam optimizer with a learning rate of 0.001 to prevent local optima, and is subsequently adjusted by employing the stochastic gradient descent (SGD) optimizer with a learning rate of 0.01. This optimizer combination helps fine-tune the model for better generalization on unseen data. The dataset is split into training, validation, and test sets, with 70% adopted for training, 15% for validation, and 15% for testing. The model performance is evaluated by utilizing metrics such as accuracy, precision, recall, and F1-score to ensure a comprehensive understanding of its effectiveness in classifying different welding states. Additionally, cross-validation is employed to further validate the model robustness.

By combining image augmentation and a center loss metric learning strategy, the application of the ResNet-18 architecture yields high accuracy in classifying different welding states and achieves an overall accuracy of over 98%, which is suitable for practical production requirements. The high accuracy demonstrates the model’s capability to reliably differentiate between various welding conditions, which is critical for real-time quality control in manufacturing environments. Key features extracted by the model are visualized by adopting guided grad-CAM and feature mapping techniques to interpret the deep learning process’s effectiveness (Fig. 7). These visualizations provide insights into the parts of the input images the model focuses on while making the predictions. The guided grad-CAM results show that the model primarily focuses on the keyhole shape and the morphology of the weld pool surface, which are critical indicators of welding quality. This focus aligns well with expert knowledge in welding, confirming that the model is learning meaningful features. Our study reveals that the deep learning framework can learn critical features from welding images, distinguishing various welding states with high precision. The utilization of guided grad-CAM provides insights into the model’s decision-making process, showing that the extracted features primarily rely on the keyhole shape and the morphology of the weld pool surface. This visualization confirms the model’s capability to focus on relevant aspects of the welding process, validating the applicability of deep learning in real-time monitoring of keyhole TIG welding. The results also indicate that the model can detect defects such as burn-through, contamination, lack of fusion, misalignment, and incomplete penetration with high accuracy. For instance, the model achieves a precision of 97% and recall of 96% for detecting burn-through defects, which is critical for ensuring the structural integrity of the welded joints. The ability to accurately detect such defects in real time can significantly reduce the need for post-weld inspections and rework, leading to cost saving and improved production efficiency.

We successfully demonstrate the feasibility and effectiveness of adopting deep learning techniques in real-time monitoring and classification of welding states in keyhole TIG welding, particularly the ResNet-18 architecture. The developed system offers significant improvements over traditional methods by providing consistent, accurate, and automated monitoring, thus enhancing the overall quality and efficiency of the welding process. Future research should focus on expanding the dataset to include more welding states and optimizing the deep learning algorithms to further improve accuracy and real-time performance. Our findings have substantial implications for the welding industry, suggesting that integrating deep learning-based monitoring systems can lead to better quality control and reduced labor costs. By automating the monitoring process, manufacturers can achieve more consistent weld quality and minimize the human error risk. Furthermore, the insights gained from feature visualization techniques such as guided grad-CAM can help refine the model and understand the key factors influencing welding quality. In conclusion, the integration of deep learning with keyhole TIG welding represents a significant advancement in welding technology, providing a promising direction for future research and industrial applications. By continuing to refine these techniques and expand their applicability, it is possible to yield even greater improvements in welding quality and efficiency, ultimately benefiting a wide range of industries that rely on high-quality welds.