Automatic Misalignment Defects Detection & Correction in PCB using SURF (Speed up Robust Features) Technique

Automatic Misalignment Defects Detection & Correction in PCB using SURF (Speed up Robust Features) Technique

Yogesh Kumar Electronics Engineering Department Greater Noida Institute of Technology

Greater Noida, INDIA

Abstract — One of the backbones in electronic manufacturing industry is the printed circuit board (PCB) manufacturing. In the PCB manufacturing process, it is an important process to detect bare board defect. Till now inspection is done manual as well as automated (for example: subtraction method, morphological method, image processing algorithms) to eliminate all 14 defects classified as breakout, short, pin hole, wrong size hole, open circuit, conductor too close, under etch, spurious copper, mouse bite, excessive short, missing conductor, missing hole, spur and over etch which comes under the two category as fatal and potential defects. Due to the fatigue and speed requirement, manual inspection is ineffective to inspect every printed circuit board. Apart from above said defects, Misalignment error can also occur during printing of PCB due to malfunctioning of conveyer belt and/or machines. Hence, this paper propose an efficient algorithm for automated visual PCB inspection system that is able to automatically detect and correct Misalignment errors on PCBs by using Surf features and morphological operation along with detection of above mentioned 14 types of errors. The results presented in the papers are quite promising for misalignment error correction and detection.

Keywords — Fatal Errors, Misalignment Errors, Potential Errors and Speed Up Robust Features (SURF).

1. INTRODUCTION

   These guidelines include complete descriptions of the fonts, spacing, and related information for producing your proceedings manuscripts.

   The performance of electronics circuit is totally dependent upon the quality of printed Circuit board. For the same, inspection of bare printed circuit board is done either by human inspectors or by automated system. But there are almost chances to make mistake by human being also its time consuming and fewer consistences. On the other hand inspection is done by automatic system detects the faults, provide fast quantitative dimensional assessments. Regarding the same, it is always preferable to inspect bare PCB in order to reduce cost on production of electronics circuit. The inspection is done by automated system removes the subjective aspects and relieves the human operator tedious boring as well as repetitive task of inspection. To get the best result from printed circuit board it is necessary to classify the inspection into two categories. (a) Fault Detection (b) Fault classification. The difference between both types of above said approach is that finding faults only in first type approach, another one classify the faults on printed circuit faults.

   Prof . S. N. Sharan Director

   Manipal University Jaipur

   On the basis of above said faults, different approaches can be classified in to three major categories (i) Reference based approach- having some techniques like Image Comparison, Image subtraction, Feature matching, Phase only method, Model based method etc. (ii) Non-reference based approach- Morphological processing, Encoding techniques etc. (iii) Hybrid based approach- Generic method etc. After detection of faults classification is done by using such algorithm like Image Processing algorithm, neural network etc.

   The basic difference among all this type of methodology is that Hybrid detection method is best one to enhance the efficiency of printed circuit board. There are basically fourteen type of fault that found on printed circuit board are as categories in to different groups-

   TABLE I. GROUPING OF FAULTS

   Sr.	GROUP	TYPES OF FAULTS
   1	First	Missing Hole , Wrong Size Hole
   2	Second	Spur, Short, Spurious Copper, Excessive Short, Under Etch negative, Conductor too close negative
   3	Third	Open Circuit, Mouse Bite, Over Etch, Conductors too close positive
   4	Fourth	Under Etch Positive
   5	Fifth	Pinhole, Breakout

   Later on these five groups converted into seven groups and then eleven groups that combine the both image processing as well as segmentation algorithm. In this each image is segmented in to four pattern provide five images with each pair of reference and test image to minimize the faults on printed circuit board. In this paper we use non-reference based approach having Morphological processing to find out fifteenth faults that is misalignment of printed circuit board.

2. BACKGROUND

Performance of electronics industry is greatly affected by production of printed circuit board including both fault free as well as faulty PCB, For the same, inspection process must be done of bare printed circuit board before production of circuit, If checking is done by human then it is great possibility to have mistaken, time taking and less efficient. On the other hand removing the faults and providing fast dimensional assessments by doing inspection through automation inspection to reduce cost spending in production of printed circuit board.

Heriansyah et al. [6] provide a methodology that find out the faults on printed circuit board through referential pixel base method and finally neural network is used for their classification. Such classification is uses binary morphological image processing and Learning Vector Quantization neural network to convert algorithm segments in to basic primitive patterns, pattern assignment, enclosing the primitive patterns as well as patterns normalization. In

Reference Image

De-noising

Binary Conversion

Test Image

this methodology 8×8 pixel size design pattern in binary form were made from eleven faulty patterns. The main advantage of this approach is that overall processing time is not affected because it did off line.

Khalid et al. [3] provide a methodology which made a process to detect and classify the fourteen type faults that considered in to five groups after conversion from binary

Image Subtraction (XOR operation)

Fatal and Potential Error Detection

Misalignment Correction (if any) (SURF Features)

Error Localization

scale into gray scale by doing comparison in between test image and reference image to avoid noise due to misalignment and uneven binary conversion on bare printed circuit board. With the help of image subtraction method separate the positive and negative image followed by NOT and flood fill operator are used for template & faulty image separately to find out the faults on PCB.

Indera Putera et al [2] proposed a methodology that is modification of Khalids method by providing the seven groups instead of five. In this process produced twenty new images with the help of four patterns by combining the image processing & segmentation algorithm. The experimental results of this technique is to detect and classify the minimum one or maximum four faults

Ibrahim [5] providing an algorithm for fault detection on printed circuit boards by using wavelet based image difference methodology. With the help of this methodology approximately faults detection are decrease by 82.11%.

Malge and Nadaf [4] use a process to detect and classify the faults on printed circuit board using a morphologica image segmentation algorithm. With the help of this method, faults can classify in certain groups by using segmentation process. This proposal uses template and test images of single layer, bare, gray scale computer generated PCBs

II. PROPOSED WORK

As there are various types of error in the PCB board after printing, in this paper proposed method majorly emphases on the misalignment errors along with potential and fatal errors. Figure 1, given below is the block diagram which represents the basic terminology of proposed method.

Fig. 1: Block diagram proposed method

Reference image corresponds to the original circuit diagram, ideally which should be exactly printed on all of PCBs. Test image corresponds to the image of the board which is printed by the machine and should be compared or tested against the reference image. Before any kind of testing procedure the reference and test images must be noise free. This noise is in existence because of the various reasons but primary it is from the imaging device used to capture these images.

To remove the noise median filter (low pass filter) is used. Similartomean filter, the median filter considers each pixel in the image in turn and looks at its nearby neighbors to decide whether or not it is representative of its surroundings. Instead of simply replacing the pixel value with the mean of neighboring pixel values, it replaces it with the median of those values. The median is calculated by first sorting all the pixel values from the surrounding neighborhood into numerical order and then replacing the pixel being considered with the middle pixel value. (If the neighborhood under consideration contains an even number of pixels, the average of the two middle pixel values is used.)

y [m, n] = median {x [i, j], (i, j) w} (1)

where w represents a neighborhood defined by the user, centered around location [m, n] in the image.As per equation

(1) given below are the required pixels. Neighborhood pixels in increasing order are: 115, 119, 120, 123, 124, 125, 126,

127 and 150.Median value: 124. De-noised images are converted in to the binary images so that all of the pixels should have the values in form of 0 and 1 only, where 1 represents the white color and 0 represents the black color.

Misalignment errors in the PCB test images are corrected using the SURF (SpeedUp Robust Features) algorithm which is based on multi-scale space theory and the feature detector is based on Hessian matrix.

Fig. 2: Neighborhood pixels of an image portion

Since Hessian matrix has good performance and accuracy. In image I, x = (x, y) is the given point, the Hessian matrix H(x,) in x at scale , it can be defined as:

(, ) = [(, ) (, )] (2)

(, ) (, )

2

Where Lxx(x, ) is the convolution result of the second order derivative of Gaussian filter 2 ()with the image I in

point x, and similarly for Lxy(x, ) and Lyy(x, ). SURF creates a stack without 2:1 down sampling for higher levels in the pyramid resulting in images of the same

resolution. Due to the use of integral images, SURF filters the stack using a box filter approximation of second

Order Gaussian partial derivatives [1]. Since integral images allow the computation of rectangular box filters in near constant time. In Figure 3 Show the Gaussian second orders partial derivatives in ydirection and xydirection.

Surf features provides the angle and scaling factor for test image with respect to reference image to be corrected and are also shown in the results and discussion section.

Fatal & potential errors are detected by XOR operations of test images with reference image. Subtraction of test image from reference image provides the potential and under printing errors, whereas the vice versa process provides the fatal and over printing errors.

Localization process involves the identification of each location of error exactly in spatial domain by the value of x and y direction. Once the values are identified then they can be easily marked as per custom indicators to distinguish them in various types.

Flow chart given in Figure 4 provides the flow diagram of the proposed method; it represents the working of algorithm to detect the misalignment errors along with other potential and fatal errors.

Steps of proposed algorithm:

Fig. 3:Gaussian second order partial derivatives in y-direction and xy- direction

1. First of all for automated inspection one must have the reference image which should be ideal with no errors, as the all-inclusive test images are compared with the reference image only.

2. Capture the test image from printed circuit board for detecting the bare defects. Imaging device environment and positioning of printed PCBs are very much important because any type of mismatching will cause the errors. Although in this paper algorithm is proposed to correct and detect these kinds of errors.

3. Before detecting any type of errors images are de- noised using median filter.

4. Misalignment errors are checked by comparing the dimension of test image with respect to the reference image and bit error rate. In case of 180o shift the dimensions remains the same but BER (Bit error rate) helps to detect the errors.

5. If there is misalignment error present in the test image angle and scaling of misalignment is calculated using SURF features.

6. Correction is done on the basis of factors obtained from the SURF features in terms of scaling and angle.

7. If there is no error or the misalignment errors are corrected then test and binary images are converted in to the binary format for subtraction.

8. To detect the fatal errors reference image is subtracted from the test image and vice versa for potential errors.

9. After detection of errors localization data is saved in excel sheet for the marking purpose.

10. The localization data is in the form of x, y locations so that any kind of custom markings can be performed by user.

Reference Image

Test Image

below. Figure 7 represents the misalignment including outliers using SURF features. Figure 10 shows the correction on the basis of errors in angles and scaling. Finally figure 11 shows the potential errors which are show in the blue color. These marked errors location are also saved in excel file automated by GUI.

RGB to Grey Scale Conversion (3D to 2D)

Image De-noising (Median Filter)

Misalignment

Misalignment Correction

Morphological

NO Processing

Binary Conversion

Image Subtraction (XOR Operation)

Error Detection (Fatal &/or Potential)

Error Localization

Fig. 5: Reference Image-I

Fig. 6: Test Image-I

Fig. 7: Putatively matched pointed including outliers.

Fig.4:Flow chart of proposed algorithm

1. RESULTS & DISCUSSION

   A user friendly GUI shown in figure 8is created for the proposed algorithm to efficiently test the results and to cater the need of non-technical end user. Given below are the test images and results by the proposed algorithm. For the test image-I results along with reference images are shown

   Fig. 8: A user friendly GUI for automated detection and correction of misalignment errors.

   Fig. 9:Misalignment factors for angle and scaling

   Fig: 12: Reference Image II

   Fig.10: Corrected misalignment errors.

   Fig. 13: Test Image II

   Fig. 11: Potential Errors.

   Fig. 14: Matching points with inliers only

   Fig. 15: Corrected misalignment errors

   Fig. 16: Fatal (Red) and Potential errors (Blue)

2. CONCLUSION

Misalignment errors can be detected easily using surf features for any value of angle and scaling but for correction of same there is constraint in proposed algorithm; up to 98% correction can be done only for the errors of angles with the difference range of 45o steps from 0o to 36o (45o, 90o, 135o, 180o, 225o, 270o, 315o and 360o) and scaling value in range 0 to 2. Correction becomes difficult beyond the above provide range. Although the proposed algorithm corrects these types of errors for any value of angle and scaling but for potential and fatal errors detection the accuracy of correction must be very high. So in a long run it can be said that the proposed algorithm provides 98% correction against misalignment errors for the ranges provide above.

Further area of improvisation can be automatic decision on the basis of errors that PCB should be corrected or not and also the range outside the given values needs to be addressed.

REFERENCES

1. Luo Juan, And OubongGwun, A Comparison Of Sift, Pca-Sift And Surf, International Journal Of Image Processing (Ijip), Vol. 3, Issue 4, Pp. 143- 152.

2. S. H. InderaPuteraAnd Z. Ibrahim Printed Circuit Board Defect Detection Using Mathematical Morphology And Matlab Image Processing Tools, Icint 2010, Shanghai, China, 2010.

3. Ijssst, Vol. 9, No. 2, May 2008, Issn: 1473-804x] N. K. Khalid, Z. Ibrahim, M. S.Z. Abidin An Algorithm To Group Defects On Printedcircuit Board For Automated Visual Inspection.

4. Malge P.S, Nadaf R.S, Pcb Defect Detection, Classification And Localization Using Mathematical Morphology And Image Processing Tools, International Journal Of Computer Applications (0975 8887), Volume 87 No.9, February 2014.

5. ZuwairieIbahim And Sayed Abdul Rahman Al-Attas Wavelet- Based Printed Circuit Board Inspection System, Ios Press, Issn 1069- 2509/05 2005.

6. Rudi Heriansyah, S. A. R.Al-Attas, M. M. Zabidi, Neural Network Paradigm For Classification Of defects On Pcb, Master Thesis, University Teknologi Malaysia, 2004.

7. Sonal Kaushik, Javed Ashraf, Automated PCB Defect Detection Using Image Subtraction Method, International Journal of Computer Science and Network (IJCSN)Volume 1, Issue 5, www.ijcsn.org ISSN 2277- 5420, October 2012.

8. [8] Theingi Aye, Aung Soe Khaing, Automatic Defect Detection and Classification on Printed Circuit Board, International Journal of Societal Applications of Computer Science Vol 3 Issue 3 ISSN 2319 8443, March 2014.

9. [9] Xiaojing Tian ,Liang Zhao, Huajun Dong, Application of image processing in the detection of printed circuit board, IEEE Workshop on Electronic Computer And Application,2014.

10. N. Yogesh Bagrecha, Quality Control Using Image Processing, et al Int. Journal of Engineering Research and Applications ISSN : 2248-9622, Vol. 4, Issue3(Version 1),pp.15- 18 ,March 2014.

11. Suhasini A ,Sonal D Kalro , Prathiksha B G, MeghashreeB S , Phaneendra H D, PCB Defect Detection UsingImage Subtraction Algorithm, International Journal ofComputer Science Trends and Technology (IJCST) Volume 3 Issue 3, ISSN: 2347- 8578www.ijcstjournal.org May-June 2015.

12. Printed circuit board defect detection using mathematical morphology and MATLAB image processing tools, Researchgate Article, DOI: 10.1109/ICETC.2010.5530052, January 2010.

13. Takumi Uemura, Gou Koutaki and Keiichi Uchimura,Image Segmentation based on Edge Detection usingBoundry Code, International journal of innovativecomputing, information and computing, Volume 7,Number 10, October 2011.

14. E. Argyle. Techniques for edge detection, Proc. IEEE,vol. 59, pp. 285-286, 2012.

15. F. Bergholm. Edge focusing, in Proc. 8th Int. Conf.Pattern Recognition, Paris, France, pp. 597- 600, 2013.