Automatic Number Plate Extraction: A Review

Automatic Number Plate Extraction: A Review

Harkamaljeet Kaur

Department of Computer Engineering Punjabi University Patiala,

India

Dr. Lakhwinder Kaur

Head of Department Department of Computer Engineering

Punjabi University, Patiala

AbstractNumber plate extraction is real life research area in the field of image processing for automatic vehicle identification. An efficient automatic vehicle identification system is useful for various applications such as automatic toll tax collection, parking system, Access control, stolen vehicles etc. Although there are numerous of number plate extracting systems available today based on different methodologies but still it is a challenging task to make an intelligent system that can be successfully implemented. In this paper, different existing number plate extraction techniques are surveyed and general steps for number plate extraction are introduced.

Keywords License plate recoginition; character extraction; character segmentation; pre-processing

1. INTRODUCTION

   The vehicle number plate consists of numeric or alphanumeric code which is the unique identification of a vehicle. As the use of vehicles is increasing every day so there is need for automatic number plate extraction system. It can be used to monitor traffic, at parking, vehicle control, toll systems etc. For the detection many automated systems have been developed, each with pros and cons for detection from images/videos. Mainly limitations are because of factors like high speed of vehicle, uniformity problem in vehicle number plate, fonts and different lighting conditions which affect a lot in the overall recognition rate. There are various algorithms used but each works under the well defined constrains. Accurate and fast detection of the number is still a challenge.

2. LITERATURE SURVEY

Rodrigo et al.[1] has made various improvements to their ANPR workflow by using intelligent heuristics, different image processing techniques as well as domain knowledge to build an ANPR system to make it capable to identify vehicles even in low resolution video frames. The advantage of the system is its real time operations, independent of special hardware, and not limited functionality in environmental conditions. Accuracy obtained by them was more than 90%.

Tsann-Tay Tang et al.[2] proposed LPR method which consists of basic steps like preprocessing, plate location, and character segmentation & recognition. Firstly, the required regions of license plate are enhanced by applying edge detection method as well as gradient-based binarization. After that horizontal projection and the corner distribution are applied. Then they performed a vertical sobel processing on the license-plate region and then the weighted-binarization method is employed which segment the characters of the license. Then skew correction is applied. Finally, a probabilistic neural network (PNN) technique is used which recognize each segmented character.

Selvanayaki, K.S et al.[3] proposed a hybrid approach for detection and recognition of vehicles. The approach gives favorable results in videos of cars recorded in good lighting conditions to overcome the limitation of systems unable to do video processing but bad weather conditions and poor lightning was still a problem.

To overcome this kind of problems Dr.Divya.K.N et al.[4] in their method used various filters to remove noise and improve contrast and sharpness of image. Recognition of image was done by using character matching and edge features like edge direction and edge gradient. Images with different color backgrounds were effectively detected. Mismatched words were identified using histogram matching using Euclidean distance but method was limited for standard fonts only.

Dr Gael et al.[5] proposed a system based on template matching to address various problems associated with recognition like different quality, image angle, complex backgrounds and fonts. This was very adaptive algorithm with good accuracy with the future provision of designing new templates with standard pixel matrix to gain accuracy level.

Samira Nigrel et al.[6] presented a system for identification of vehicle by number plate recognition using genetic algorithms with main application in traffic systems. In this extraction was done using edge detection algorithms and segmentation using filtering and morphological algorithms and finally genetic algorithm for recognition of plate characters. The method was simple and efficient with satisfactory results under various conditions.

Serkan Ozbay et al.[7] presented application software designed for recognition based on extraction using edge detection algorithms . For segmentation they used smearing, filtering and some other algorithms. And statistical based template matching is used for the recognition of plate characters. Algorithms show superior performance and further can be improved to extend them for multinational number plates.

Kuang et al.[8] proposed a novel method for license plate detection. The key steps in their method are image preprocessing, license plate detection and license plate confirmation. Initially the noises are removed and the different license plate forms are processed using image pre processing techniques. And then, rough detection of license plates is done using the cascade AdaBoost classifier. Finally, the gradient images are converted to binary and then the connected component analysis is adopted which removes false plates.

Tang et al. [9] focused on the binarization of poor illumination for practical implementation in car license plates system. In their work used mathematic morphology filter as a low-pass filter which improves the illumination of license plate. For eliminating illumination, average niter, Gaussian filter and median filter are used to filter the plate image respectively. Then they converted the gray-scale image to binary image using the local threshold which can be calculated by convolving the image with the coefficient of illumination.

1. IMAGE ACQUISITION

   Due to technological advancement image acquisition process can be achieved at low cost. High performance and high definition cameras available easily makes it easy. Different systems takes an image as input captured by any source which may be any satellite image or image captured by camera etc. with aim to produce successful and effective output irrespective of the obstacles such as vegetation, terrain conditions, blurred images, poor luminance etc.

   Tian et al. [10] presented a algorithm for license plate detection in traffic surveillance environment which is complex. Their detection process has mainly three steps that are decomposition, modeling and inference. Firstly the decomposition of license plate is done into the constituent characters. To extract the characters in images the maximally stable extremal region detector is used. On these then the CRF models are constructed. Then they integrated the spatial and visual relationships of the characters and integrate with CRF that is in the form of probability distribution. Finally, the exact bounding boxes of license plates are estimated based on belief propagation inference.

   III. GENERAL OVERVIEW

   As every system is designed for specific application, so the design of system depends on the function and environment conditions. Number plate recognition systems can generally be divided in four common steps: (1) Image acquisition (2) number plate detection (3) character segmentation and (4) character recognition. As shown in fig.1 the first step is to capture image of ehicle. It looks easy but may be difficult in some cases like images of moving vehicles, distant images. it should be in such a manner that the different components of vehicle should not be missed especially the number plate. The success and accuracy of recognition depends highly on the effectiveness of number plate detection and segmentation of each character. Systems can follow different approaches to locate number plate and extract the number of vehicle. Some commonly used approaches are discussed in this paper.

   Image Acquisition

   Image Acquisition

   Number plate detection

   Number plate detection

   Character segmentation

   Character segmentation

   Character recognition

   Character recognition

   Figure.1- A general overview

   A. Image Pre-processing

   In pre-processing requirement is enhancement and restoration. it involve processes which improve the quality of image by eliminating noises, reversing the damages, de- blurring to recover the original sharpness of an image, highlighting edges of the images.

   1. Gray scale processing

      The captured image can be altered by various factors like: camera distortion, noise, excessive relative motion of camera or vehicle etc. The result is the downgrading of a captured vehicle image which adversely influences the further image processing So before the main image processing, pre- processing of the image should be done. It includes- converting RGB to gray for noise removal, border enhancement for brightness. Colored images are also complex in space and time, therefore it is necessary to convert them to gray scale to reduce time and space complexity. The basic idea behind gray conversion is to eliminate hue and saturation by not affecting its luminance. For this, we compute the threshold of an image by using the suitable gray scale value. This separates the object of interest from background. Thresholding is important to provide sufficient contrast of an image such that, varying level of intensity between foreground and background are considered. Gray scale conversion enhances the quality of an image for later computational processes. Gray scale images consist of different ranges of gray values; from 0 to 255. MATLAB function rgb2gray converts a RGB image to gray scale image.

   2. Median Filtering

      It comes under the category of non linear filters. It changes the gray value of the pixels to the median of the gray value of surrounding pixels. We use a 3×3 mask and calculate the corresponding gray value of each pixel using the 8 neighboring pixels. This helps in noise removal. Median filtering gives advantages such as no reduction in contrast since output values are its neighborhood values, boundaries remain unchanged. Median filters are very useful in the presence of impulse noises also called salt and pepper noise because of its appearance as white and black dots superimposed on an image.

   3. Histogram equalization

      This technique is used to enhance color distribution in unclear images. It reduces the clustered pixel values in an image .it also help in even distribution of images with the light and dark areas.

   4. Smoothing linear filters

   The output of this filter is obtained by the averaging the pixels contained in the neighborhood of the filter mask. Therefore they are also called as averaging filters. Their other name is low pass filters. The concept behind these filters is straightforward. We replace the value of each pixel in an image by the average of the gray levels in the neighbors defined by the filter mask. It results in an image with less sharp transitions in gray levels. Generally random noise mainly consists of sharp transitions in gray level. The straightforward application of smoothing is noise reduction.

2. NUMBER PLATE DETECTION

   Mainly the number plate detection algorithms fall under different categories based on different techniques. To detect vehicle number plate the most common factors to be considered are-

   (1). Plate size: plates can be of variable sizes in different vehicle images.

   1. Plate location: to identify where the plate is located in the vehicle.

   2. Plate background: plates sometimes have different background colors which are based on vehicle type like government vehicle number plate in india have yellow background color.

   3. Screw: A plate will have screws which may be mistaken as character.

   We can extract a number plate can be extracted by using image segmentation method.

   1. Edge Detection

      Edge detection is main method of feature detection or number plate detection. In common cases the result of applying this algorithm is an object boundary with connected curves. This method is difficult to apply in complex images. In such cases it sometimes result with object boundary only, not connected curves. There are many edge detection algorithm or operators like Canny, Laplacian, Differential, Canny-Deriche, Sobel, Prewitt and Roberts Cross. The prewitt and sobel edge detectors are better for delineating edges than the Robert edge detector. The prewitt masks are easier to implement than the sobel masks as they are simple. But sobel masks have superior noise-suppression characteristics. To detect wide range of edges in image we use canny edge.

   2. Hough Transform

      This is a technique for feature detection. Initially it was used for line detection. Now it has been extended to find position of different shapes like circle, square, oval etc. Therefore we can use it to detect number plates from the image based on their rectangular shape.

   3. Blob detection

      This is used to detect points or regions which have varying level of brightness or color in comparison to surroundings. This approach is mainly used to find complimentary regions that are not detected by edge detection or corner detection algorithms. The commonly used blob detectors are Laplacian of Gaussian (LoG), Determinant of Hessian (DoH),

      Difference of Gaussians(DoG), principal curvature and maximally stable extremal regions detector.

   4. Mathematical morphology

   Mathematical morphology is based on lattice theory, set theory, random functions, and topology. It can be applied to digital images as well as spatial structures. Firstly it was developed to process binary images only and later on improved to work on gray scale functions and images. It contains operators like -Erosion, dilation, opening, closing.

   The methods discussed above are common methods for plate detection. There are various other approaches available also like sliding concentric window using region of interest, image binarization, fuzzy discipline based approach, probabilistic neural network.

3. CHARACTER RECOGINITION

   Character recognition is used to identify and converting image text into editable and readable text. To perform character recognition we should first perform character segmentation with high accuracy. Otherwise character recognition will have errors. Methods such as image binarization, vertical and horizontal projection, CCA etc are used for segmentation of characters from the image.

   1. Character recognition using Template matching

      It is useful when we have characters of fixed size. It have numerous other applications like face detection, medical image processing etc. It is further classified in two parts: template based matching and feature based matching. Feature based approach is used if template image has strong features otherwise we can use template based approach. To adjust all characters with uniform size we can use linear normalization algorithm. In matching process we move the template image to all possible regions of source image and compute a numerical index which indicates the accuracy of template match with the position. Then a correlation coefficient is calculated between the input character matrix and template character matrix. Character with the maximum value of correlation coefficint are considered to match the most and produced as output.

   2. Character recognition using optical character recognition Optical Character Recognition algorithm can also be used to recognize the vehicle number. We invert the image obtained after the number plate extraction that is convert all the white pixels to black and vice versa. This results in white text and black background. Before applying this we separate the individual lines in the text by using line separation process. This adds the each pixels value in a row. When the resultant sum of row is zero we assume that no text pixel is present in a row and if greater than zero it means the text is present in row. The first resultant sum which is greater than zero indicates the start of the line and the end of line is represented by the resultant sum zero after it. These values- start and end of the line are used to crop the initial line in the text. The process continues to separate all the lines of text. Once the lines are separated, the same process is applied column wise to separate individual characters. These are then stored in variables. By using OCR we can now compare the each individual character with the complete alphanumeric

   database. It actually uses correlation method for matching individual characters and at last the number is identified and stored in a variable in string format.

   S.no	Authors		Techniques	Limitations	accuracy
   1.	Serkan Ozbay and Ergun Ercelebi		Edge detection, morphological operations, statistical based template matching	Designed for	Extraction
   				identification	(97.6%)
   				of Turkish	Segmentation
   				number	(96%)
   				plates only	Recognition
   					of characters
   					(98.8%)
   					Overall-
   					97.46%
   2.	Dhiraj Gaikwad and Pramod Borole	Y. B.	Edge detection, optical character recognition	OCR method sensitive to alignment	Satisfactory
   3.	Selvanayaki, K.S. and Rm. SomaSundaram		Blob Detection and Extraction Connected component analysis	Dont work well for govt vehicles as they carry logo and emblem information	Not reported
   4.	Dr.Divya.K.N & Dr.Ajit Danti		Edge detection, edge gradient, histogram matching	Different fonts	Performs effectively
   5.	Samira Nigrel, Akash Ashokan ,Bidisha Barua		Edge detection, morphological, genetic algorithm	Not reported	92.5%

   S.no	Authors		Techniques	Limitations	accuracy
   1.	Serkan Ozbay and Ergun Ercelebi		Edge detection, morphological operations, statistical based template matching	Designed for	Extraction
   				identification	(97.6%)
   				of Turkish	Segmentation
   				number	(96%)
   				plates only	Recognition
   					of characters
   					(98.8%)
   					Overall-
   					97.46%
   2.	Dhiraj Gaikwad and Pramod Borole	Y. B.	Edge detection, optical character recognition	OCR method sensitive to alignment	Satisfactory
   3.	Selvanayaki, K.S. and Rm. SomaSundaram		Blob Detection and Extraction Connected component analysis	Dont work well for govt vehicles as they carry logo and emblem information	Not reported
   4.	Dr.Divya.K.N & Dr.Ajit Danti		Edge detection, edge gradient, histogram matching	Different fonts	Performs effectively
   5.	Samira Nigrel, Akash Ashokan ,Bidisha Barua		Edge detection, morphological, genetic algorithm	Not reported	92.5%

4. RESULTS OF SOME TECHNIQUES INTRODUCED

5. CONCLUSION

Number plate extraction has different phases and accuracy of each phase dependent on previous phase. Presently, the techniques are not general. Some of the techniques are for specific countries. Different style and formats of number plate as well as the environment conditions under which the systems are designed to operate are the main challenges. Maximum accuracy i.e. 97.46% is achieved using morphological operations and template matching. Factors that affect the performance of algorithms are various font styles, moving of capturing device, illumination, alignment.

REFERENCES

1. Ajanthan T., Kamalaruban P., and Ranga Rodrigo, Automatic Number Plate Recognition in Low Quality Videos. Published in ICIIS,2013,Industrial and Information Systems ,8th IEEE International Conference on Date of Conference: 17-20 Dec. 2013 Page(s): 566 571

2. Chao-Ho Chen, Tsong-Yi Chen, Min-Tsung Wu, Tsann-Tay Tang, Wu-Chih Hu, License Plate Recognition for Moving Vehicles Using a Moving Camera Published in 2013, Ninth International Conference on Multimedia Signal Processing(MSP) and Intelligent Information Hiding Page(s): 498 500I.S. Jacobs and C.P. Bean, Fine particles, thin films and anisotropy exchange, Magnetism,Volume 3,G.T. Rado and H. Suhl, Eds. New York: Academic, 1963, pp. 271-350.

3. Selvanayaki, K.S. and Rm. SomaSundaram, Hybrid Approach for Detection and Recognition of Vehicles Journal of Computer Science 2015, 11 (2): 304.314R

4. Divya.K.N, Dr.Ajit Danti, recoginition of vehicle number plates and retrieval of vehicle owners registrarion details published in international journal of innovative search issn:2321-1156.

5. Dabas and Dr Savita Gael, Vehicle Registration Plate Recognition System Using Template Matching published in ieee, nov 2013.

6. Samira Nigrel, Akash Ashokan, B. Barua, Slesha Patil and Vaibhavi Patil Identification of Vehicle by Number Plate Recognition Using Genetic Algorithm published in International Journal of Advanced Research in Computer and Communication Engineering ,ARCCE,Issue 2,Vol. 4, February 2015

7. Ergun Ercelebi and Serkan Ozbay, automatic vehicle identification by plate recognition presented in World Academy of Science, Engineering and Technology International Journal of Computer, Electrical, Energetic, Electronic and Communication Engineering Vol:1, No:9, 2007

8. Runmin,Ruolin Wang, Nong Sang and Xiaoqin Kuang, A Novel License Plate Detection Method for Complex Scenes Published in IEEE, 2013, 7th International Conference on Image and Graphics 978- 0-7695-5050-3/13 $26.00 © 2013 IEEE DOI 10.1109/ICIG.2013.69

9. Yi Wangi , Bin Fangi , Li-jun lan , Hui-Wu Lu,Yuan-Yan Tang, Adaptive Binarization: A new approach to License plate characters segmentationPublished in the 2012 ,International Conference on Pattern recoginition and & wavelet analysis, ian, 15-17 July, 2012

10. . Bo Li, Bin Tian, Ye Li, and Ding Wen, Component-Based License Plate Detection Using CRF Model published in IEEE 2013,under the topic Transactions on intelligent transportation systems, VOL. 14, NO. 4, DECEMBER 2013

11. Hajare Tai, Harmale Priyanka, Jadhav Neha and Mali Kanchan, automatic vehicle identification by plate detection published in IJARS, International Journal of Advanced Research in Computer Science and Software Engineering 5(3), March – 2015, pp. 1046-1050

12. The Selvanayaki, K.S. and Rm. SomaSundaram, Hybrid Approach for Detection and Recognition of Vehicles

13. Chirag patel, dipti shah and atul patel Automatic Number Plate Recognition System (ANPR): A Survey published in IJCA, International Journal of Computer Applications(0975 8887) Volume 69 No.9, May 2013