IC Label Inspection Sub-System Design for Product Quality Conformation Based Template Matching.

IC Label Inspection Sub-System Design for Product Quality Conformation Based Template Matching.

Mkongwa Kefa Gideon1,a, Zhijun Pei1,b, Jun Zhang1,c

1Department of Electronic Engineering, Tianjin University of Technology and Education Tianjin, China, 300222

Abstract.

IIIIJJJJ

IIIIJJJJ

EEEE

EEEE

RRRR

RRRR

This paper represents architectural design for an industrial inspection sub-system for product quality conformity ,IC label inspection,that combines the image processing and pattern recognition algorithms (template matching) with the embedded firmware design for the actuation system.Template matching based normalized cross correlation is the algorithm of choice for this paper,this is due to its higher accuracy and shorter computational time of the image processing library used,The implementation of the algorithm is done on visual basic (VB 6.0)SDK,after several algorithm tests done on MATLAB.Having developed an application on VB environment,makes the architecture more comfortable for real time application.

1. Introduction.

   Template Matching in image processing is a process of comparing region of an image with template,we have region of choice called reference(template),which is compared with

   the region in an incoming image through a series of searching routine, (a process known as template matching) It is widely used in many applications, such as object tracking and monitoring in military applications, satellite image monitoring, image registration ,medical image analysis for medical diagnosis,industrial product inspection systems,weather forecasting,simple object

   recognition etc. Matching is commonly done in many methods and still many researchers are working to increase accuracy and reduce computation time[1,3].Generally, some methods like;Mean Absolute Difference, Mean Square Error, Luminance similarity, Correlation,Contrast similarity and Hausdorff distance are used to find the similarity between objects.Number of works has been proposed to detect the object along with feature extraction such as edges, regions and corners and then to locate the target by combining the features but these works have high computational complexity[1,3] .Cross correlation is an important and basic tool in image processing,which correlates two signals to find the match, it being suffering from scaling,rotation and illuminance variation makes most researchers use alternative methods to encounter the problems.In this paper the problem of illuminance variations and rotation have higher significance of study, the algorithm developed worked with higher efficiency for real time application

   TTTT

   TTTT

2. Challenges with NCC.

   Object inspection has been suffering from larger size of the template upon searching and localization by the template matching algorithms,further object searching to some algorithms depends on good texture and illumination[4],(SIFT-whose measure of similarity depends on nearest neighborhood with computational complexities for complex images),similarly edge detection by some

   detection methods like Harris detector proves failure on poor illuminations,another problem with the NCC method is impossibility to define the absolute threshold in the similarity score because the score on an image got by NCC method is dependent on the image content. In order to overcome this problem, an improvement to the NCC comparison method is presented here.

3. Proposed architecture for NCC.

   Design started with simple mathematical evaluation of the NCC using simple matrices to just search selected matrix elements. The NCC equation is defined by;

4. Illumination invariance.

   Appearance of an image may change from object to object due to varying pixel intensity levels as a result of clutter,occlusion or illumination invariance.to take control of the NCC algorithm for objects with illumination invariance is of great challenge.to get a robust and fast results the changes in object's appearance should be minimized as much as possible when imaging an object.nevertheless sometimes problems like occlusion,clutter or defocus can not be avoided.

   In our case we considered grabbing object's images with an acquisition device that will bring gray scale images into image processing subroutines,illumination invariance was solved by

   k l taking the following measures:

   f (u i, v j)t(i, j)

   k

   k

   l

   l

   r ik j l

   f ^2(u i, v j)

   ik j l

   IIIIJJJJ

   IIIIJJJJ

   EEEE

   EEEE

   RRRR

   RRRR

   where f and t are incoming image and template respectively.so the algorithm design in based on the above equation,the template will move over the incoming image and evaluate normalized correlation coefficient at each point of an image covered by a mask.correlation coefficient is normalized between -1 and +1.where +1 indicates the exact match and the degree of mismatch increasing as the values moves below +1.The region whose normalized correlation coefficient qualifies the threshold set by by the industrial tolerance to conformity,says to meet the industrial quality requirements[10].

   For inspection to be done,the incoming object should first be aligned in the same axis as the reference image,this is done by affine transformations to eliminate problems encountered by incoming object rotation or angular displacement and scaling.

   The absolute definition of pass or fail depends on the way the NCC is performed,for our case the template is divided into smaller subblocks instead of searching for the whole template at once.

   Illuminance variations was referenced to camera

   1. The imaging device was enclosed to avoid external illumination changes.

   2. proper fixing of the imaging device by maintaining constant distance between the lens and object's surface.

      TTTT

      TTTT

   3. images were taken in gray scale into image processing subroutines.

5. Object's rotation tolerance.

   Object rotation is another hindering factor for NCC as we have seen before.for this case the affine transformations(using projective matrices)were deployed to align object's image with reference before searching algorithm starts.observation showed as objects moves along the line there is higher possibility of angular displacements due to mechanical rotation(rotation through the IC center) of the conveyor belt,precaution for this was by confining all objects in a specific container , further affine transformation subroutine is included in the algorithm,that will align objects as a result of angular displacement,as the objects were confined in the container susceptibility to higher displacements was avoided and hence small tolerance to rotation was provided.

   position from the object with varying lens capacities and brightness over the image capture

   x' cos sin

   x x

   0

   0

   setup environment where the distance of the lens from the object was a driving force for better performance of the algorithm[2].

   y' sin cos y y0

   Where x,y are the coordinates of rotated

   object,x0,y0 original object coordinates, displacement ,x',y' are the coordinates after

   transformations.based on the confinement of the ICs was given tolerance of [-0.5 +0.5] radians.

6. Threshold selection by K-Mean.

As the matching is template based the criteria for threshold selection was don using statistical evaluation of the Euclidean distance of each correlation coefficient as a measure of deviation from the particular class.

and later retrieved into the processing algorithm SDK,visual basic 6 .0 was used for the complete image processing subsystem design with a user friendly interface with which the user migrates over the interface to have access to the reference image and incoming image in separate windows[7],command to start or stop processing alternately various parameters as a result of

d0 (x, y)

(x x )2 ( y y )2

processing are displayed and finally print the processing result summary.

0

0

0

0

By evaluating centroid for each randomly selected classes of data in every iteration.

Each IC represents one object with two points ,the correlation coefficients for two block templates of the label r(logo,part number).based on the observation the initial centroid data points were selected for three kinds of correlation results,accepted r(0.865,0.513),poor quality r(0.605,0) and rejected r(0,0).random selection of the initial centroids based on the closeness of the data to +1,from the correlation results obtained upon testing,the algorithm is executed until convergence as follows:

Iterate until stable (= no object move group):

1. Determine the centroid coordinate

   EEEE

   EEEE

   RRRR

   RRRR

2. Determine the distance of each object to the centroids

   IIIIJJJJ

   IIIIJJJJ

3. Group the object based on minimum distance (find the closest centroid).

Out of which the selection of the threshold was found best at r>=r(0.7,0.5) for pass,and r(0,0) for rejection,while the in between class was found to have poor qualities.

The classification is supervised because of the fewer number of data used for algorithm testing.

1. Industrial setup,algorithm synthesis.

   On every search iteration the mask moves over the object and the correlation coefficient of each region covered by the mask is obtained,the higher the correlation coefficient the closer is the similarity of the object to the reference one.second level algorithm was tested using a very powerful industrial machine vison library,Halcon.The images are grabbed by the most sensitive industrial camera"microvision_mv1300um" with which gray scale image is taken into the image processing subroutine,model creation subroutine selects the template using special features like coordinates, to mark the region and hence extract features of the particular region of interest(mask).Grabbed image is first registered

   Figure 1.Reference image.

   TTTT

   TTTT

   Figure 2.Incoming image slightly rotated.

   Figure3.incoming image,Toshiba.

   Figure 4.sample results of a found match of input image figure 2.

   In this paper we only inspect the logo"ATMEL" and the part number "ATF16V8B-15PU" availability and conformity.incoming image with both of this regardless of the orientation should be in position to undergo inspection,the inspection system has three kinds of results based on the K- mean classification,accepted,poor quality and rejected.

   Accepted :there is a match as per the tolerance set by the manufacturer.

   poor quality :refers to some misalignments character spacing errors or poor printing . rejection :refers to either missing a print out or the print out is completely wrong.

   It is named a subsystem because inspection targets only on the label informations though there are a number things for complete IC inspection like pins

   arrangement and fracture on the surface of an integrated circuit(IC).

   Visual basic SDK is linked to the embedded system comprised of the optical sensors and the image acquisition device "Microvision_MV- 1300um" with external trigger[7],the results after VB synthesis fed to the firmware (microchip- program) as an interrupt through RS232 interface and the actuation system being represented by the LED blink on port 2 of an AT89S52 micro controller not included in the diagram.processing summary analyzes the actions to be taken for each processing results.

   self explaining Algorithm flow chart,

   IIIIJJJJ

   IIIIJJJJ

   EEEE

   EEEE

   RRRR

   RRRR

2. Results and discussion.

   Several tests have been done to end up with the results shown on this paper.

   Through various tests observation shows that many ICs which were taken for testing have different qualities,this helps to prove that some ICs are pirated an effect which affects the business performance of the technology owner,and hence the users who buys counterfeit ICs for the sake of serving leading into system instabilities in their designs.

   The observations that helps to prove that many ICs taken for testing are pirated is because of the behaviors seen in the poor labels,same label with different character spacing and height refer figures here under;

   The part numbers are misaligned;

   Figure 5.misaligned label.

   Figure 6.misaligned label.

   The label has all the required details but the alignment of the label disqualifies the product to the market as a result of poor business performance of the technology owner due to either unethical manufacturers poor printing or poor inspection systems used.

   TTTT

   TTTT

   So to generalize the inspection systems for this cases of figure 5 and figure 6,an IC whose label has such misalignment issues is rejected by the inspection system as a poor product quality despite it having the said label details.this is done by taking the reference to that of figure2,which doesn't allow misalignments.

   Figure 7.perfect label.

3. Conclusion.

   NCC performs better on the search and localization with higher scores despite the poor character alignments as it is shown in the figure 5 and 6.

   Correlation coefficient greater than 0.7,0.5 is chosen to qualify for the conformity for our case and as our curiosity bases on the inspections for quality conformation character alignment is also very sensitive.

   The design of the subsystem for IC label inspection was successful with an efficiency of more that 90%.

   The design has diverse application areas as it has been listed in the introduction part of the paper and the concept can be used to develop a number of real time service systems to be used in our for both manufacturing industries and custom applications.In spite of the available most

   powerful algorithms used for recognition systems NCC is proven to be the most accurate one with better results to date.The software developed can be used in industrial subsystems for quality management and ensure customers with a conformed product quality by increasing competition and business assurance.Figure 4,sample results cropped from figure 5. which shows the final user interface design

   displaying results of the inspection sub-system.log in page.

   TTTT

   TTTT

   Figure 8.Log in for authorized users.

   IIIIJJJJ

   IIIIJJJJ

   EEEE

   EEEE

   RRRR

   RRRR

   Figure 9.GUI.

   Figures 8 and 9 shows computer's screen shot displaying the users' LOG IN &GUI

4. Acknowledgment

   This work is supported by the National Natural Science Foundation under Grant No.61071204.

5. References.

1. Tim Morris"computer vison and image processing"

2. H. Chang, J. Zhang, New metrics for clutter affecting human target acquisition, IEEE Trans. on Aerospace and Electronic Systems, vol. 42, no. 1, pp. 361-368, Jan. 2006.

3. International Journal of Theoretical and Applied Computer Sciences

   Volume 1 Number 1 (2006) pp. 934

   (c) GBS Publishers and Distributors (India) http://www.gbspublisher.com/ijtacs.htm

4. David Marimon and Touradj Ebrahimi"efficient rotation-discriminative template matchng"signal processing Institute(ITS) 2007.

5. SergiosTheodoridis,Konstantinos Koutroumbas"pattern recognition 4th ed"

6. Rafael C.Gonzalez,Richard E. Woods"digital image processing 3rd ed"

7. Diane Zak"programing with microsoft visual basic

   6.0 enhanced ed"

8. Brian R hunt,Ronald L.lipsman"A guide to matlab beginers and experienced users"

9. BiPOM electronics.inc"Microcontroller to sensor interfacing techniques"

10. Safaa L Diab"Designing visual inspection system for quality characteristics"mechanical eng' research- 2011.

11. Ebooksclub.org"digital image processing using matlab.pdf"

IInntteerrnnaattiioonnaall JJoouurrnnaall ooff EEnnggiinneeeerriinngg RReesseeaarrcchh && TTeecchhnnoollooggyy ((IIJJEERRTT))

IISSSSNN:: 22227788–00118811

IIIIJJJJ

IIIIJJJJ

EEEE

EEEE

RRRR

RRRR

TTTT

TTTT

VVooll.. 22 IIssssuuee 11,, JJaannuuaarryyrr– 22001133