A STUDY ON DETECTING THE WELD SPOT FOR BUILDING AN AUTOMATIC WELDING ROBOT

The automatic robots have become an important trend in the industry to improve the productivity and performance such as introducing welding robots for the welding process. The automatic welding robot helps increase production efficiency and reduces risks and labor costs. This study proposes an advanced method by incorporating the YOLOv8 deep learning network into an automatic welding robot. The task of the YOLOv8 model is to detect and monitor welds in real time through a camera system. This helps automate the welding process by providing information about the position and shape of the weld points. Welding coordinate values are transmitted and the welding tip is adjusted to target specific welding points, creating an efficient and precise automated welding process. The results are useful for the development and application of automatic robot and artificial intelligence (AI) in the field of welding and industrial production. By combining welding technology and artificial intelligence, the welding industry will continue to progress and modernize for contributing to the sustainable development of global manufacturing.

1             Introduction

The automatic robots (AR) have become an indispensable part of the production processes to improve the performance and productivity such as introducing robot for industrial applications [1], using welding robots for welding process [2, 3], developing UAV for navigation duty [4]. AR shows good characteristics of flexibility, precision, safety, and  efficiency for industrial applications and human services [5]. The automatic welding robot (AWR) is not only an advancement in technology but also an effective approach to increase production productivity and reduce risks and labour costs such as using welding robot for working in underwater environment [6], developing mobile welding robot for nuclear working condition [7], and introducing welding robot for unstructured environment [8]. Despite significant progress, there are still challenges that need to be overcome in the development of welding robot control systems. The welding robot has been received many concerns of researchers to develop control methods for its applications such as developing systems based on motion control [9], 3D simulation [10], and machine learning [11]. These methods all have their advantages but still face certain limitations when applied in real industrial environments. One of the most important challenges is the accuracy of the welding process [12]. Although modern control systems can be tuned to achieve high levels of precision, maintaining stability and uniformity under all operating conditions remains a problem. In addition, the applicability of welding robot control methods in real industrial environments is also an important issue. Working conditions are often varied and not always stable, and this can affect system performance. The ability to train and deploy these systems also requires significant investment on the part of the business, from employee training to integration into existing production processes. The robot control has brought many benefits to the manufacturing industry; the continuing research and development are needed to overcome challenges and optimize performances of the actual production process.

One of the biggest duties of automated welding robots is the ability to accurately identify weld points on work surfaces. Accuracy in determining the welding point not only affects the quality of the final product but also ensures the safety and performance of the welding process. Many object-tracking algorithms based on deep learning have been proposed. Initially, the YOLO algorithm achieved impressive speed but not high accuracy, especially when detecting small objects and locating overlapping objects. To overcome these limitations, the following versions of YOLO have been developed, each aimed at improving detection accuracy and efficiency. YOLOv2 has been introduced innovations such as anchor boxes and multi-scale training, enhancing the ability to accurately position and handle objects of different sizes [13]. YOLOv3 has been improved object detection by using a feature pyramid network (FPN) and multiple detection scales providing better detection at different resolutions [14, 15]. These improvements have boosted YOLO's performance in many scenarios. YOLOv4 brings more advanced architectural modifications [16] such as the CSPDarknet-53 [17] and PANet backbone [18], improving real-time object detection accuracy. YOLOv5 has been focused on optimizing the architecture to speed up inference while maintaining accuracy [19]. YOLOv6 and YOLOv7 have been continued to improve by exploring new approaches [20, 21]. YOLOv8 has been introduced to industrial application with effective innovations [22]. YOLOv8 is expected to leverage the strengths of previous versions and explore new approaches to enhance real-time object detection.

Basically, YOLOv8 is not only used as an object detector; it is also the theoretical core of the perception module. By using a segmentation variant of YOLOv8, the model generates pixel-level weld seam masks, allowing us to formulate weld point extraction as a geometric inference problem rather than relying solely on bounding box localization. The choice of this approach is based on the assumption that weld seams exhibit visually distinguishable textures and shapes, which can be learned through convolutional feature representations.

This study advocates the application of the YOLOv8 model, recognized as an advanced solution in the field of object detection, to solve the important task of automatically identifying weld points during the welding process. This effort appears at the intersection of welding technology and artificial intelligence, offering a promising path to revolutionize conventional approaches to welding processes and increase automation in the manufacturing sector. By exploiting the capabilities of advanced object detection techniques contained in the YOLOv8 model, the work aim to overcome the limitations of traditional manual methods and conventional automatic systems in weld spot recognition.

Furthermore, the proposed methodology relies on a calibrated camera-to-robot transformation model. We assume that once the weld seam is segmented, geometric features extracted from the mask track such as its centerline or boundary pixels that can be mapped into the robot coordinate system through homogeneous transformations. This assumption allows the segmented weld points to be used directly for trajectory generation.

Integrating advanced machine learning algorithms with welding technology not only improves the accuracy and reliability of weld spot detection but also paves the way for unprecedented levels of efficiency and productivity in industrial environments. The importance of this research lies in its ability to catalyse transformational changes in the manufacturing landscape. By leveraging the power of artificial intelligence, the paper strives to streamline and optimize welding processes, thereby minimizing inefficiencies, minimizing errors, and improving overall quality in operations production activity. Through the seamless combination of welding technology and advanced machine learning methods, we aim to redefine the industrial welding and manufacturing paradigm, ushering in an era of automation, precision, and high performance. Finally, the research seeks to make tangible contributions to improving efficiency and quality in industrial welding and manufacturing processes. By unleashing the synergy between welding technology and artificial intelligence, the work strives to empower manufacturers with innovative tools and approaches that drive operational excellence, fuelling sustainable growth sustainability and propelling the industry towards unprecedented levels of competition and success.

2             Materials and methods

2.1     The structure of an automatic welding robot

Figure 1 illustrates a basic structure of an automatic welding robot. During the welding process, an important part is detecting the weld so that the robot can perform the welding process accurately and effectively. To do this, the robot has equipped a camera system, integrated with an advanced image recognition algorithm, YOLOv8. This camera system has an important task in identifying welds on pipe surfaces. When a weld is detected, the camera will send its coordinates to the welding robot to continue the work. The two main types of welds for automated welding robots have identified to handle including vertical welds and horizontal welds. This helps optimizing the welding process for both cases. Once identifying the type of weld, the robot arm will automatically control the position of the weld and start the welding process. The submerged arc welding is one of the preferred welding methods, especially in industrial applications, because it offers high performance and uniform welding quality [23]. In particular, this method also helps the camera not be affected by disturbing agents such as welding fumes or circuit lights during the process of monitoring the weld. The automated welding robot system is not only a combination of technology and automation but also a prime example of innovation in the modern welding industry.

Fig. 1. A basic structure of an automatic welding robot

2.2     YOLOv8

You Only Look Once (YOLO) is structured by three main components including the main trunk, the neck, and the detection head [24, 25]. Each component plays a crucial role in achieving accurate and efficient object detection. YOLOv1 is known for its groundbreaking real-time object detection capabilities. It utilized the Darknet-19 backbone architecture, enabling simultaneous prediction of multiple bounding boxes and class probabilities. YOLOv2, also known as YOLO9000, introduced enhancements in both accuracy and speed by incorporating anchor boxes for better localization and scaling predictions across different objects. YOLOv3 is further improved accuracy and speed by utilizing a feature pyramid network (FPN) for multi-scale feature extraction, resulting in better detection of small objects and improved bounding box regression. Then came YOLOv4, a significant advancement over its predecessors, featuring backbone architectures like CSPDarknet53 and incorporating techniques like Mish activation function and PANet for improved performance [26]. YOLOv5 took a different approach, focusing on simplicity and efficiency with custom lightweight backbone architectures and simplified feature extraction methods. Looking ahead to hypothetical versions, YOLOv6 could potentially explore more efficient backbone architectures like EfficientNet and incorporate advanced feature extraction techniques such as attention mechanisms or transformer layers [27]. YOLOv7 might experiment with novel backbone architectures optimized for object detection tasks and explore advanced feature extraction methods like self-attention or capsule networks. YOLOv8 leverages state-of-the-art backbone architectures, such as Vision Transformer (ViT) or other transformer-based models adapted for object detection [28]. Feature extraction methods would likely involve transformer-based approaches, allowing for more efficient capture of long-range dependencies and semantic information. Additional key features might focus on addressing specific challenges in object detection tasks, such as handling occlusions or fine-grained object recognition, by incorporating specialized modules or attention mechanisms tailored to these tasks. The structural diagram of YOLOv8 remains akin to the design of CSP in YOLOv7. The backbone, the first part of the YOLOv8 network, extracts features from input images, often constructed using deep convolutional layers. The neck, connecting the backbone and head, combines information from feature maps of different resolutions to enhance object recognition ability through convolutional and pooling layers. Finally, the head predicts the location and class of objects present in the image using convolutional and pooling layers, followed by thresholding and non-maximum suppression operations to specify and mark objects on the output image. Through continuous iterations, the YOLO series aims to enhance speed, accuracy, and efficiency in object detection tasks, introducing innovations in architecture and feature extraction techniques [29, 30].

2.3     Dataset

Our dataset, comprising 980 images captured by the camera, offers a comprehensive representation of various welding scenarios encountered in real-world environments. Figure 2 exhibits a diverse spectrum of lighting conditions and environmental contexts, ensuring a rich and nuanced dataset for analysis. Within these images, weld spots exhibit a range of size ratios, reflecting the variability inherent in welding processes across different contexts and applications. To facilitate precise object detection and classification, we employ sophisticated labeling software, meticulously annotating each image to identify and categorize the weld spots. Horizontal weld lines are denoted as "Weld_X", while vertical weld lines are labeled as "Weld_Y", ensuring unambiguous identification of these critical features within the dataset. Upon completion of the annotation process, the labeled images undergo further organization to facilitate effective training and evaluation of object detection algorithms. Utilizing a systematic approach, it has partitioned the dataset into distinct subsets, adhering to a predefined ratio of 2:1 for the training and test sets respectively. This thoughtful division ensures that our models are trained on a diverse range of data while also providing a robust mechanism for assessing their performance on unseen samples.

Fig. 2. Some samples in the dataset

3             Results and discussions

To evaluate the results with the weld joint, we use the testing environment as shown in Table 1. Overall, the experimental setup is well equipped with high-performance hardware and software components, providing a suitable environment for conducting comprehensive evaluations of weld joint detection using the YOLOv8 model.

Table 1. Experimental environment

Figure 3 shows the results of identifying horizontal and vertical welds. The test data set also included the images collected under different environmental conditions, thereby creating a diverse and complete data set. Specifically, for the horizontal weld point "Weld_X", the accuracy ranges from 80% to 90%. This shows that the YOLOv8 model can accurately and reliably identify and classify horizontal welding points across a variety of environmental conditions. However, for the vertical weld point "Weld_Y", the accuracy ranges from 70% to 80%, which is lower than that for the horizontal weld point. However, although the accuracy may not be as high, it still shows that the model is capable of identifying and classifying longitudinal weld points at an acceptable level. Overall, the experimental results on the test data set have demonstrated the impressive performance of the YOLOv8 model in identifying weld points under different environmental conditions. This is an important step forward in applying artificial intelligence to the welding industry, helping to optimize production processes and enhance product quality.

Fig. 3. Results of identifying horizontal and vertical welds

With positive results from the experiments, we conducted real-time performance validation of the model. Figure 4 shows that for horizontal welds, the model achieved an impressive accuracy of up to 88%. This is a remarkable achievement, highlighting the model's ability to accurately and reliably identify and classify horizontal weld points, even in real-time environments. When evaluating longitudinal welds, the results remain at a stable level, but are lower, fluctuating between 60% and 70%. This suggests that, in some cases, the model may have more difficulty recognizing longitudinal weld points in a real-time environment. Although the accuracy may not be as high, it still shows that the model is capable of providing useful information and decision support in real-life situations.

Fig. 4. Results of identifying horizontal and vertical welds in real-time

Fig. 5. The coordinates of the welds are continuously updated

Table 2 The welding point coordinates for the welding robot

The coordinate output algorithm for the welding robot operates along the x and y axes, thereby controlling the robot to move precisely to the desired position on the Oxy coordinate system. As illustrated in Figure 4, the YOLOv8 algorithm is employed to detect weld seams within the camera's frame. In this specific instance, the system successfully identified a total of three welds, including two horizontal welds (Weld_X) and one vertical weld (Weld_Y). Once detected, the coordinates of the welds are continuously updated along the x and y axes within the camera's working space. This ensures that the system can accurately track and pinpoint the location of each weld, regardless of any changes that may occur during the welding process.

To obtain accurate welding coordinates for the robot, the system uses the pixel-based segmentation output directly from the YOLOv8 model. As illustrated in Figure 4, each weld seam is segmented into a continuous vertical or horizontal region, and every pixel belonging to that region is extracted. These values are then listed in Table 2, where each entry represents a pair of (X, Y) pixel coordinates captured from the camera frame. The parameters in Table 2 are not manually chosen but are automatically generated based on the following tuned settings of the detection pipeline:

-       Confidence threshold: set to 0.7, ensuring that only high-confidence weld detections are processed for coordinate extraction.

-       IoU threshold: set to 0.5 to balance sensitivity and precision when separating adjacent weld seams.

-      Segmentation mask resolution: retained at the YOLOv8 default scale to preserve boundary details of the weld region, which directly affects the accuracy of pixel coordinate extraction.

Coordinate sampling strategy: for each segmented weld seam, all foreground pixels are collected instead of selecting only edges or centerlines. This allows the robot controller to compute a more stable and noise-free weld trajectory.

These tuned parameters ensure that the pixel coordinates accurately represent the real weld geometry. As a result, the robot receives a dense and consistent set of spatial points for each weld (Weld_Y, Weld_X_1, and Weld_X_2), enabling precise motion planning along the Oxy coordinate system.

The robot utilizes this data to guide its welding arm to the precise location of each weld on the workpiece. This process not only minimizes errors and enhances precision in the welding operation but also increases the robot's operational efficiency. With the capability to detect and process coordinates continuously and automatically, the welding system can function stably and effectively, ensuring the highest quality welds.

To further validate the effectiveness of the proposed method, comparative experiments were conducted with several state-of-the-art weld detection algorithms, including YOLOv5, Faster R-CNN. The results show that YOLOv8 achieved superior accuracy in weld seam detection while maintaining a significantly faster inference speed. These improvements allowed the system to generate more stable and precise pixel-based segmentation results, which directly enhanced the reliability of coordinate extraction for the robot. This comparative evaluation confirms that YOLOv8 offers a practical and high-performance solution for real-time robotic welding applications.

In summary, the integration of the YOLOv8 algorithm with the coordinate control system of the welding robot has created an intelligent and efficient automated welding process that meets the stringent requirements of modern industry. Figure 4 and Table 2 demonstrate the feasibility and effectiveness of this system, optimizing the production process and improving product quality.

4     Conclusions

This article has researched and tested the application of the YOLOv8 model in identifying welding points in the welding process. The results show that the YOLOv8 model is capable of accurately and reliably identifying and classifying horizontal welding spots while providing stable results in a real-time environment. Although the accuracy in identifying longitudinal weld points may be lower, the model is still reliable and provides useful information in real-life situations. This research marks an important step forward in the application of artificial intelligence to the welding industry while providing a solid basis for optimizing production processes and enhancing product quality. This research encourages the development of smart welding technology and the application of artificial intelligence in the manufacturing industry, thereby making meaningful contributions to economic and social development.

These authors thank Hanoi University of Industry, 298 Caudien Street, Hanoi 100000, Hanoi, Vietnam.

The author declares no conflicts of interest.

Van-Long Trinh and Cong-Duy Do conceptualized the study and developed the methodology. Software was prepared by Quang-Tu Ngo and Viet-Hoi Tran. Validation was performed by Van-Long Trinh and Tien-Sy Nguyen, and formal analysis and data curation were conducted by Van-Long Trinh and Cong-Duy Do. Investigation was carried out by Van-Long Trinh and Quang-Tu Ngo, with resources provided by Van-Long Trinh and Viet-Hoi Tran. The original draft was written by Cong-Duy Do, and review and editing were done by Van-Long Trinh. Visualization was performed by Tien-Sy Nguyen and Viet-Hoi Tran. Supervision was provided by Van-Long Trinh. All authors have read and agreed to the published version of the manuscript.

Data are contained within the article.

There are no supplementary materials to include.