Survey Paper on Visual Inspection of a Mechanical Part using Machine Learning

Survey Paper on Visual Inspection of a Mechanical Part using Machine Learning

Priya Charles

HOD, Department of Electronics & Telecommunication Dr. D Y Patil Institute of Engineering Management & Research, Akurdi

Pune, India

Simran Gupta

Department of Electronics & Telecommunication Dr. D Y Patil Institute of Engineering Management &Research, Akurdi

Savitribai Phule Pune University Pune, India

Niraj Bhadoria

Department of Electronics & Telecomunication Dr. D Y Patil Institute of Engiineering Management &

Research, Akurdi Savitribai Phule Pune University

Pune, India

Pooja Satpute

Department of Electronics & Telecommunication Dr. D Y Patil Institute of Engineering Management &

Research, Akurdi Savitribai Phule Pune University

Pune, India

Abstract Product manufacturing industries produce machine parts in vast quantities. Quality assurance and control of these parts is necessary to maintain the standard defined by industry. Traditional methods of inspection using quality control inspector leads to inconsistency and imprecise decisions. Thus, Current manual inspection procedure is erroneous. The purpose of this research is to overcome the flaws in the manual inspection using automated visual inspection system. Collecting data for processing is done with the help of Image acquisition system, which captures image of the part on production line. This captured image is pre-processed and feature extraction is performed resulting in generation of feature vectors. Using machine learning model for defect classification, we train ML algorithm with these feature vectors to classify the defects found during the inspection. defect detection and other quality control actions are performed based on the reference part. The detection of defects at an early stage helps in increasing quality control factor and smoothing the process of production. automated visual inspection system saves time and is efficient. This survey definitively answers the question regarding the defect detection of machine parts.

KeywordsVisual, Inspection, Machine Learning, defect, detection.

1. INTRODUCTION

   Product manufacturing industries have automated production line which produce required product/parts at an astonishing rate. These parts are to be visually inspected before further processing. This inspection process can be manual or automated. The manual inspection requires an inspector to check the defects in the part, quality control inspectors can adapt quickly to changing light conditions. The automated inspection system is rapid and consistent over specified quality details, manual inspection of the machine part is impractical due to fast moving parts on the production line, it is also subjected to inaccuracy and inconsistency. To overcome these issues a computer based automated visual inspection technique is developed. For inspection of these machine parts we use image processing and machine learning models. The

   recognition of a machine part in production is quite different from object detection.

   Visual inspection of machine parts is carried out at an early stage to ensure the quality of products. The defects in the products are classified into surface defects and damages. Surface defects include dents, burr, scratches, texture etc. Damages to the machine part may include cracks and breaks on the part. The automated inspection helps in improving the quality of product with increasing production rates. Thus, proving to be highly efficient where machine parts are massly produced.

   Existing methods of visual inspection follow basic steps of image acquisition, object location and object quality measurements. Here we acquire the images of the machine parts form the production line with the help of mounted cameras that are further processed and matched with pre- established quality control descriptions.

2. LITERATURE SURVEY

   TITLE 1: Machine parts recognition and defect detection in automated assembly systems using computer vision techniques

   Authors: P.Arjun, T.T.Mirnalinee

   Abstract: The following paper suggests computationally efficient 2D computer vision-based approach to recognize the machined and detect damaged parts in the assembly line. The system acquires images of the part and the image data is pre- processed for background subtraction, scaling and noise removal to generate data suitable for further processing. The contours of the part are then extracted to figure out the shape of the part. Here the data extracted is compared with the model shape. Defects are identified by the similarity measurement with model shape. The inspection at the early stage helps in smoothing the production and saves cost and time.

   TITLE 2: Automated Visual Inspection of Glass Bottle Bottom with Saliency Detection and Template Matching Authors: Xianen Zhou , Yaonan Wang, Changyan Xiao , Qing Zhu , Xiao Lu, Hui Zhang, Ji Ge, and Huihuang Zhao.

   Abstract: The paper describes apparatus based on machine vision for real time bottle bottom inspection. The framework used is saliency detection and template matching. They first locate the bottom and divide the region of interest into three parts, central panel region, annular panel region, annular texture region. Saliency detection is employed for the central panel region, multiscale filtering is adopted for the annular panel region and combination of template matching and multiscale filtering is used to detect defects in the annular texture region. The results of these three regions are combined together to get the quality description for bottom of the test bottle. The experimental results of the following apparatus are highly efficient in comparison with other conventional methods.

   TITLE 3: Real-Time Visual Inspection and Rejection Machine for Bullet Production

   Authors: Anugool Thamna, Pornsak Srisungsitthisunti, Surangsee Dechjarern

   Abstract: The aim of this paper is to improve quality control of bullet cartridges M193 during manufacturing process, thus replacing the current human inspection process. The automated visual inspection system consists of a bullet feeder and high-resolution cameras to capture images of bullets moving on the production line for digital image processing. The processed image will be further analyzed to identify any possible defects such as dark spots, crack lines, deformation, unevenness, dimensional check etc. The software with the help of image analysis sends a command to reject the defected bullet with help of a PLC controller. This automated visual inspection system is much faster in speed and accurate. The database of the different defects and damages will help in determining and eliminating the cause of the defects.

   TITLE 4: Automatic Visual Inspection of Printed Circuit Board for Defect Detection and Classification

   Authors: Vikas Chaudhary, Ishan R. Dave and Kishor P. Upla Abstract: Inspection of printed circuit board (PCB) is an important process in the electronic manufacturing industry. The defects of the PCB are detected and classified in an order. In this paper all of the classified defects are identified using referential inspection approach. The algorithm used involves image registration, pre-processing, segmentation, defect

   TITLE 5: Vision-based robotic system for picking and inspection of small automotive components

   Authors: Oleksandr Semeniuta, Sebastian Dransfeld and Petter Falkman

   Abstract: As size of the component decreases the visual inspection becomes a challenging task. Handling of small components to avoid noise and uncertainies, their inspection requires advanced image analysis. The paper focuses on concept of multi camera or multi pose inspection of a star washer. The process involves picking the part from the feeder and then inspection is carried out in the dedicated assembly. The tasks performed at the inspection rig are part identification, machine learning based classification, circular object-based image analysis and star washer teeth segmentation.

3. EXISTING METHODOLOGY

   In the following section we shall see the existing methods for automated visual inspection.

   In [1], The steps followed in the automated visual system are: image capturing, image pre-processing, contour extraction, area normalization, centroid distance calculation, similarity matching, machine part recognition and defect detection. Segmentation, Binarization, Background subtraction and object extraction, image resizing, removal of holes, noise removal are objectives of image pre-processing.

   Shape feature extraction is applied on the processed image which generates the contour of the processed image by assuming the contour pixel point farthest from the shape centroid and tracing the contour pixels in the clockwise direction which generates the contour of the part in the image.

   The recognition of machine part and its inspection requires a reference shape of the machine part. Reference shape dataset is created from the flawless machine parts. These referenced shapes are used further in the template matching process. The machine part recognition is the first step which identifies the machine parts. In the defect detection step we identify the defects by observing the deviations in the results. The feature vectors of the machine part (P) and referenced shape (Q) are compared using correlation coefficient metric. The coefficient r measures the similarity between the two -machine part (P) and reference shape (Q) as shown in equation (1).

   detection and classification. The algorithm functions well even

   10( , )( , )

   2

   2

   if the input image is rotated, scaled, etc. with respect to the

   (

   , ) =

   ,=

   reference image, i.e. the algorithm is rotation and scale in-

   (1( , ) ) (1(

   , )2)

   variant. The proposed algorithm is able to identify defects in any possible appearance and requires minimal time to inspect

   =0

   =0

   (1)

   the image which makes the algorithm quite robust. The efficiency of the algorithm is tested with subjecting the algorithm to PCB images with different defects of varying appearances and severity levels.

   here ( , ) are feature vectors of the machine part and

   reference shape respectively and ( , ) are its mean values

   of the shapes P and Q. The value for correlation coefficient is bounded between -1 to +1.

   In [2], The ROI is extracted using a combination of circle detection with size, as the bottom of bottle is standard circular

   object and has fixed size. 2IHT algorithm is used to obtain the ROI of the bottle bottom. The algorithm consists of two steps, radius detection and centre detection. 2IHT is efficient and requires less memory than traditional Hough circle detection. After localization they are classified in two measurement regions, central panel and annular region. The annular region

   pixels. The PCB image has three different parts viz. wiring tracks, soldering pads, holes. Histogram thresholding and mathematical morphology operations are performed to segment the image. Upper and lower threshold values are used to get the wiring and solder pad segments.

   is further divided into annular panel region and annular texture region.

   =

   1, 95 < < 140

   { 0,

   (3)

   Defect detection in these three regions are carried out separately.

   Annular texture region uses template matching based defect

   =

   1, > 140

   {0,

   (4)

   detection which includes three steps: regular texture localization, template extraction and template matching. Defects usually appear as deviation in the correlation coefficient of the template matching process.

   Defect detection in annular panel region is based on Multiscale mean filtering. MMF has three basic steps, Projection and mean scale filtering, comparing projection profile and filtering result, obtaining maxima for final defect classification. The final result is computed by

   The zero region of the solder pads represent holes, which can be extracted using basic morphological operations.

   Defect detection is the final stage where the segmented image of test and template are subtracted in order to detect the defect. The subtraction process generates positive defect (PD) and negative defect (ND).

   =

   (5)

   =

   () = arg DminDmax

   {(, )}

   (2)

   Where, i = soldering pads, wiring tracks, holes.

   (6)

   Where {D (y, bi)}.is the difference between the projection profile and the filtering results. Values of R(y) are high in the defective region and low in the normal region.

   Central panel defect detection is carried out using saliency detection methods viz. Region Growing Geodesic Saliency (RGGS) and Region Growing Euclidean Saliency (RGES). RGGS uses geodesic distance with region growing algorithm whereas RGES uses Euclidean distance to improve the speed.

   In [3], Bullet cartridges are passed through the sensors mounted on the conveyor. The sensor sends signal to software program instructing the camera to capture images. Camera signals the PLC to start the inspection process and reject bullets that have defects.

   As bullets are made of brass and have reflective surface, the position of the light source and bullet are crucial in defect detection. To ensure even lighting bar type light source was used.

   Software program was developed using NI VIEW development module in LabVIEW.

   In, [5] Functional prototype for star washer picking and inspection is implemented. The process involves, star washer identification, Circular object analysis, Classification of star washers orientation and Inspection of star washer teeth.

   Star washer identification thresholds the test image using Otsus method for optimal thresholding. The following output has maximum variance between the black and white pixels. This is used to separate the connected components and final result is a labelled image.

   Circular object analysis uses polar representation to obtain the profile along the segment i.e. the metallic part of the image. As we are interested in the metal washer and not the hole in the centre.

   Star washers orientation is acquired using trained Machine Learning models. The models are trained on the feature vectors generated by the reference image and images in which washers are disoriented.

   A high-resolution image of star washer is analysed to segment each star tooth, the analysis results are two angles ,

   2

   In [4], The proposed algorithm employs five stage process for classification of defects that occur during production of PCB. Image registration is the first stage where the PCB is scanned

   =

   2

   (7)

   in order to get test PCB image. Image registration helps in removing the rotation or translation variations in the test image with respect to template. Extraction of features from test image

   = ( + )

   (8)

   and template is done at this stage. Feature extraction process is time consuming therefore we use Features from Accelerated Segment Test (FAST) algorithm. This algorithm takes less time as compared to other techniques.

   Next stage is image pre-processing which removes noise and enhances the quality of the image.

   Image segmentation is carried out after pre-processing stage. Segmentation represets image in separate parts or set of

   Using these angles each star tooth can be segmented into an

   individual image. This gives detailed information about the start and end of each tooth.

4. PROPOSED METHODOLOGY

   Our proposed system aims on inspecting a machinery part to determine the defects present in the part. If the defects are present, it will then classify it into two categories depending

   on whether or not the defect can be eliminated i.e. re- machining or rejection of the part. In case of no errors-

   Our main focus lies on the defects mentioned below:

   • Number of holes present (NH)

   • Threading present/ absent (MH)

   • Broaching (BR)

   • Surface defects (SD)

   • Cracks (CR)

   • Hole alignment (HA)

     Fig 1. Proposed system.

     For this we are using a camera that can capture images at four different angles i.e. 90, 180, 270 and 360. We are using the supervised learning algorithm to train our model. Wherein, we capture images of the part and classify them into training and test datasets. This algorithm will be applied on the datasets to train our model, the performance evaluation will be measured with suitable metric such as confusion matrix, logarithmic loss, Area under curve (AUC), F measure. It will then help in determining the accuracy of the model and its capability in classifying the defects in appropriate classes. Defects which can be rectified are sent for re-machining.

5. ADVANTAGES

   For better productivity and quality results in the production industry the need for improvement in quality inspection is required. Automated visual inspection helps in enhancing this inspection process further. Compared to manual inspection the automated inspection has following advantages:

   • Consistent and accurate

   • Increased speed of defect detection

   • Quick adaption to different products and their specifications

   • Able to function for longer period of time.

   • Can be programmed and remotely controlled

6. CONCLUSION

A non-contact visual inspection system with minimum manual intervention. Flexible over wide range of products and their defect characteristics. Supervised learning algorithm trains the model which then classifies the input image into given classes. The output can lead to re-machining, rejection or acceptance of machinery part. The automated visual inspection is consistent and precise, leading to an increase in quality assurance factor.

REFERENCES

1. P. Arjun, T.T. Mirnalinee Machine parts recognition and defect detection in automated assembly systems using computer vision techniques Rev. Téc. Ing. Univ. Zulia. Vol. 39, Nº 1, 71 – 80, 2016

2. Xianen Zhou, Yanonin Wang, Changyan Xiao, Qing Zhu, Xiao Lu, Hui Zhang, Ji Ge, Huihuang Zhao Automated Visual Inspection of Glass Bottle Bottom with Saliency Detection and Template Matching IEEE Transactions on Instrumentation and Measurement, 2019 IEEE.

3. Anugool Thamna, Pornsak Srisungsitthisunti, Surangsee Dechjarern, Real-Time Visual Inspection and Rejection Machine for Bullet Production 2018 IEEE.

4. Vikas Chaudhary, Ishan R. Dave and Kishor P. Upla, Automatic Visual Inspection of Printed Circuit Board for Defect Detection and Classification IEEE WiSPNET 2017 conference.

5. Oleksandr Semeniuta, Sebastian Dransfeld, and Petter Falkman, Vision-based robotic system for picking and inspection of small automotive components 2016 IEEE International Conference on Automation Science and Engineering (CASE) Fort Worth, TX, USA, August 21-24, 2016