The Design of an Intelligent Augmented Reality Welder Training System

The Design of an Intelligent Augmented Reality Welder Training System

Yeong-Do Park

Department of Advanced Material Engineering Dong-eui University

Busan, Korea

Jinseok Seo

Division of Digital Contents Technology Dong-eui University

Busan, Korea

AbstractThe objective of this research is to design an AR (Augment Reality) GMA (Gas Metal Arc) welding training system, which can overcome weakness of the pre-existing augmented/virtual reality based welding training systems. The newly designed system satisfies requirements for presence, hand-eye coordination, versatility of welding postures, diversity of shapes of weldments, and welding diagnostics. In order to design an ideal welding training system, following 3 approaches are applied; AR, neural network, and scriptable contents.

Keywords Welder Training; Augmented Reality; Neural Network Based Optimization;Welding Simulation

1. INTRODUCTION

   The most of the metal-based structures are bonded using welding between homogeneous or dissimilar metal materials. At this time, the welds of the two materials have different physical properties from those of the raw materials, and they have a great influence on the stability of the final structure according to the welding quality of the welds. In particular, passive welding by human welders takes up a large part in the assembly processes of shipbuilding and automobiles

   Recently, with the development of automation processes, the welding process has also been converted to automation systems. However, the welder is still manually welding in the areas where automation is difficult. Therefore, the stability of the welded structure is greatly influenced by the technical skills of human welders, so there is an urgent need for human welders having a long experience and expertise. However, training of skilled welders requires at least two years of training period, and expensive welding equipment and consumables are also required. Particularly, there are many difficulties in educating skilled welders because of the presence of disadvantages such as welding fumes. In addition, in most of the training courses currently conducted, it is necessary to conduct a non-destructive inspection process after welding to determine the integrity of the welding. It is time consuming and repetitive because of the difficulty of immediate feedback to trainees. In order to solve these problems, it is essential to develop a welding training system based on virtual reality or augmented reality.

   The several welding training systems using virtual reality has been developed all over the world, and some of them are now commercialized. However, most of them utilize computer simulated virtual space, and the screen that welders see is limited by virtual graphics objects, including weldments. In the end, it is difficult to realize the cooperation between a trainee and welded parts in the eyes and hands. In

   addition, due to the nature of the pre-built virtual environment, the diversity of the shapes of weldments is limited.

2. REQUIREMENTS FOR AN IDEAL WELD TRANING SYSTEM

   The effectiveness of augmented reality based training systems has already been demonstrated in several research and practice cases. Successful examples include military, surgery, rehabilitation, painting and so on. In this study, the technical requirements that an ideal welding training support system should have are classified into 5 as follows, and a method to solve this problem is devised and a prototype system is designed.

   1. High Realism

      Trainees want to weld while they are looking at an actual weldment, not the virtual objects simulated by computer graphics. In addition, the simulation should be based on actual data measured directly in the real world so that the arc image, sound, etc. are almost similar to the actual welding situation.

   2. Hand-Eye Coordination

      In human work using hands, coordination of visual information and hand movement is very important. The positions of the physical hands and a welding torch felt by trainees and the positions in the virtual environment must match exactly.

   3. Versatility of Welding Postures

      In the industrial field, the welding postures that require the skilled technologies of welders are vertical and upright, but most virtual reality based training systems only support the below viewing posture.

   4. Diversity of the Shapes of Weldments

      Training on various shapes of weldments should be possible. In most practical environments, the shape of the weldment changes frequently.

   5. Welding Diagnotics

   Immediate feedback is required depending on the dynamically changing welding conditions during a welding operation. A comprehensive diagnostic report after welding is essential for an effective training system.

3. RELATED WORKS

   Fig. 1. ARC+ System [2]

   Welding training systems using virtual reality or augmented reality have been developed in the United States, Canada, and France [1] and can be categorized into three approaches. ARC+ [2] system developed in Canada features a virtual environment using a 3-D screen (See Fig. 1). The motion of a welding gun using a special marker can be tracked and a 3-D bead shape representation is also possible. The images are implemented using an HMD (Head Mounted Display) and welding fumes are also visualized. The welding operation is represented by five variables such as motion straightness, arc speed, stick-out, work angle, and travel angle. Additionally, it supports task analysis and feedback. However, there is a disadvantage that the realistic interaction during welding training is not high because the variety of representation for weldments and the quality of bead visualization are poor.

   Fig. 2. CS WAVE [3]

   Fig. 3. SimWelder [4]

   CS WAVE system developed in France [3] is a non- immersive welding simulator that does not use a welding helmet unlike ARC + (See Fig. 2). The movement of a welding torch is designed to operate horizontally and vertically on a computer screen. The position of the welding torch is tracked using gyroscopic and ultrasonic techniques. It provides tools for analyzing results such as work speed and welding angle, and a portable version is also available to support the convenience of carrying.

   SimWelder [4] was developed by EWI (Edsion Welding Institute), Navy Manufacturing Technology, and General Dynamic Electric Boat (See Fig. 3). The most significant feature of this product is the shape database that is composed of the actually measured data of the GMA (Gas Metal Arc) weld bead. In addition, proprietary mesh simulation technology is used for real-time representation of various welding processes. Especially, in welding training using plastic mockups, this system visualizes the virtual arc and bead shape through the display mounted inside the helmet. They also used neural network to optimize the final weld bead for various welding conditions. There are also an FCAW Simulator and DesignAR developed in Korea, but it is still a basic stage compared with the systems introduced above.

   Fig. 4. Augmented reality welding helmet [5]

   An example of applying augmented reality is a helmet for welding support developed in Germany [5] (See Fig. 4). The high-resolution HDR camera is mounted on the top of the helmet, and the arc image is filtered to minimize a glare, and facilitate the observation of the arc shape. However, this system is not used for education but it is used as an auxiliary means to improve the weldability and to improve the efficiency of welders.

   TABE I.

   System	Type	Req. 1	Req. 2	Req. 3	Req. 4	Req. 5
   ARC+	IVR	high	high	low	med	med
   CS WAVE	NIVR	med	med	high	med	high
   SimWelder	IVR	high	med	med	med	high
   FCAW Sim	Sim	low	low	med	med	med
   DesignAR	Sim	low	low	med	high	low
   ARWelding Helmet	AR	high	high	med	med	high
   Our Objectives	AR	high	high	high	high	high

   The above case studies can be classified into non- immersive virtual reality system which does not wear HMD, immersive virtual reality system which wears HMD, and augmented reality system, according to implementation methods. A non-immersive virtual reality system allows for a relatively high hand-eye coordination because welders can see their hands and the actual welding torches directly. On the other hand, an augmented reality based system is more suitable for the welding training support by providing a high realistic feeling and hand-eye coordination. The Table I shows the results of a comparison between the above- mentioned research examples and our study objectives.

4. PROPOSED SYSTEM

   Fig. 5. The overall structure of our proposed system

   The final goal of this study is to propose an intelligent GMA welding training support system based on immersive augmented reality. The main features and differentiations of our system can be described as follows.

   1. Augmented Reality System

      The augmented reality based system has 3 advantages such as high realism, hand-eye coordination, and diversity of shapes of weldments, as compared to virtual reality based systems.

      In virtual reality systems, trainees are trained by looking at virtual simulation results in a virtual space created by 3D computer graphics technology. However, in augmented reality systems, they can observe actual objects (welding torch and their hands), thereby provided a more realistic environment.

      Another advantage of augmented reality systems is that it provides high hand-eye coordination. In virtual reality- based systems, since workspace, weldments, a welding torch, and the hands of trainees all exist in the virtual space, it is not easy to coordinate simulation results observed by trainees and an actual welding torch and hands. On the other hand, augmented reality systems provide high level of hand-eye coordination and higher training effects because trainees can see a real image of workspace, welded parts, a welding torch, and their hands.

      In order to develop training contents based on virtual reality, 3D computer graphics designers and computer programmers have to develop new contents every time the structure of the weldment changes. However, in augmented reality systems, trainees can be trained on actual welded parts, so various types of trainings can be done at low cost.

      In this study, a prototype augmented reality welding helmet is made by combining a camera with a video-through HMD with LCD screens, a welding helmet, and markers for optical tracking. The position and orientation of the user's head can be tracked to render a virtual image at a desired point in time.

      In addition to the markers attached to a welding torch and the welding helmet, markers are attached also to weldments so that virtual beads and arcs can be visualized at correct

      positions even if a trainee freely change the position and direction of weldments.

   2. Artificial neural network-based total posture weld simulation

      Most virtual reality training systems use a simple method for visualization of the shapes of weld beads, which makes it difficult to diagnose the detailed welding results. So, this disadvantage can hinder the training effects. In this study, it is aimed to make detailed and various diagnosis by simulation and visualization of optimized shapes of beads in real time using artificial neural network technique.

      In addition, in this study, the shapes of welding beads in various welding postures such as vertical, horizontal, top view, and bottom view is managed in a database system and the artificial neural network enables welding trainings in various postures.

   3. Script-based training content authoring

      To enable training on a variety of weldments, it is not possible to use only augmented reality-based systems. Though augment reality can reduce the cost of modeling virtual 3D weldments and coding computer, it is impossible to simulate welding without knowledge of the structure of weldments and welding postures.

      In this study, we can define the structure of weldments and welding postures using only a simple XML based script, so that various training contents can be written easily and quickly without the help of professional computer programmers. An XML-based scripting language has an advantage that it is easy to learn even for beginners, and it is very easy to improve it later as a GUI (Graphical User-Interface) based authoring tool because the structure is clear and simple.

   4. GMA Welding Database

      In order to simulate welding in augmented reality contents, data measured in actual welding must be constructed in a database. The parameters (for data acquisition) set in this study are current, speed, working angle, running angle, and contact tip to work distance (CTWD). By collecting about 50 kinds of actual welding data by changing these parameters, it is enough to make the augmented reality contents necessary for the basic education course.

      Fig. 6. Before and after filtering of an arc image

      The result of welding for each combination of above parameters is the shapes of beads and arcs. These results are captured into real images and managed in a database using a 3D shape measuring device and a high-resolution camera. Especially, the images of arcs generated by welding processes can be filtered by image processing algorithms. The filtered images are used for rendering virtual arcs, later. Fig. 6 shows an example before and after filtering of an arc image that welders observe in actual welding.

   5. Neural Network Based Welding Results Estimation

   Even if there is a DB that has accumulated a lot of welding data, it is not possible to simulate all the welding conditions. In this study, we try to solve this problem by using neural network. The neural network estimator, trained using the data obtained from dozens of experiments, predicts the welding results suitable for new input data including current, speed, angle, CTWD, etc.

5. CONCLUSION

Fig. 7. Requirements and approches in this study

As defined in Chapter 2, the ideal welding training support system needs to have high realism, hand-eye coordination, versatility of welding posture, diversity of the shapes of weldments, and welding diagnostics. In this study, we tried to apply the approaches of augmented reality, artificial neural network, script language to meet these requirements. Fig. 7 shows how each approach is applied to meet those requirements.

Although welding training institutes are expanded, it is difficult to supply practical materials and skilled professional welding educators. Therefore, through the spread of the augmented eality welding training system proposed in this study, it is expected to be utilized as an educational tool for cultivating experienced welder manpower in a short time.

REFERENCES

1. T. Heston, Virtual welding training in a virtual environment gives welding students a leg up, Fabricator, March 2008.

2. C. Choquet, ARC+ (Registered Trademark) and ARC PC Welding Simulators: Teach Welders with Virtual Interactive 3D Technologies, 2011.

3. L. D. Dalto, et al., Cs wave: Virtual reality for welders, IEEE VR, 2005.

4. W. Bleck, U. Reisgen, O. Mokrov, E. Rossiter, and T. Rieger, Methodology for Thermomechanical Simulation and Validation of Mechanical Weld-Seam Properties, Adv Eng Mater 12(3):147152, 2010.

5. P. Tschimer et al., A concept for the application of augmented reality in manual gas metal arc welding, Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR02) 2002.