Comparative study of License Plate Recognition

Comparative study of License Plate Recognition

Varsha K. Hadke TSSMs Bhivarabai Sawant

College of Engg & Research, Narhe Pune, India

AbstractLicense Plate Recognition (LPR) is the most interesting and challenging area of research due to its importance to a wide range of commercial application. It is known that the number plates differ in shape and size. License plate recognition (LPR) is an image processing technology used to extract vehicle information from their license plates. There are three modules in license plate recognition system. 1) Detection 2) Character segmentation 3) Character recognition. There are many techniques for license plate detection, each having its own advantages and drawbacks. The basic step of license plate detection is localization of number plate. This paper compares morphological method, histogram based method and mathematical morphology methods of license plate detection. Character segmentation is done by using connected component and thresholding method. Character recognition is done by template matching method. The aim of this paper is to study and evaluates some of the most important license plate recognition algorithms.

Keywords License plate localization, Morphological operation, Character segmentation, License plate recognition.

1. INTRODUCTION

   License plate recognition (LPR) is an image processing technology used to identify vehicles by their license plates. License plate recognition systems have wide range of application such as traffic maintenances, stolen cars, and automatic electronic Toll collection system and law enforcement. Reading automatically the license number means transforming the pixels of the digital image into the ASCII text of the number plate and comes under a broad area of optical character recognition.

   All vehicle world-wide should have its license number – written on a license plate mounted onto its vehicle body. There are many kinds of license plates used in the country. License plate used in India mostly consists of 8, 9, or 10 character [1]. License plate is the ID of an each vehicle. There are many algorithms proposed for LPR. Some of them have good accuracy with more complexity than others. Some of those algorithms are computationally intensive. However, selecting one of them based on some criteria such as execution time, success rate of character recognition of the algorithm, and its accuracy in different situation is a challenging problem. Based on that purpose of this paper is to study and to evaluate some license plate detection (LPD) algorithm and compared them.

   Pawan. K. Ajmera TSSMs Bhivarabai Sawant

   College of Engg & Research, Narhe Pune, India

   Most of the LPR systems are based on image processing techniques and character recognition systems.

   Each LPR system consists of three modules, license plate detection, character segmentation, character recognition.

   License plate recognition applies image processing and character recognition technology to identify vehicles by reading their license plate numbers. LPR process consists of two main stages: 1) locating license plates and 2) identifying license numbers of that license plate. In the first stage, located license plate candidates are determined based on the features of license plate. Features commonly employed have been derived from the license plate format and the alphanumeric characters consisting license numbers. The features regarding license plate format include shape, symmetry, height-to-width ratio, color, texture of grayness, spatial frequency, and variance of intensity values [2]. Character features include line, blob, the sign transition of gradient magnitudes, the aspect ratio of characters, the distribution of intervals between characters, and the alignment of characters. Most of the images processing techniques for the license plate detection are based on neural network, Gabor transform, Hough transform, and Ada- Boost models. But there are some common features affecting on performance of these algorithms including:

   • Lighting conditions such as cloudy weather, night working hours, reflecting sunlight or cars taillight or front light.

   • Complex background which effects on speed of detecting the real region of plate.

   • Damaged or dirty license plates that causes to gain fault identification numbers.

   • Varying view angle and efficient distance between camera and moving car.

   The rest of this paper organized as follows. In section II, pre-processing techniques are described. Different approaches of license plate detection are discussed in section III. Segmentation and recognition module are described in details in section IV and V, respectively. Section VI discusses experimental results and the paper concludes with section VII.

2. PREPROCESSING TECHNIQUES

   1. Gray scale conversion

      Conversion of color image into a gray image. The red, green and blue components are separated from the 24-bit color value of each pixel to calculate the 8-bit gray value using the formula:

      = 0.59 + 0.30 + 0.11 (1)

   2. Median filtering

      It is used to replace the gray value of each pixel with the median of the gray values of its neighbor. Here used 3 × 3 masks which help in getting eight neighbours of the pixel and their corresponding gray values. It helps to remove the salt-and-pepper noise.

   3. Edge enhancement

   A new nonlinear spatial domain edge enhancement method based on gray tone morphological operator. The method employs a morphological gradient as it can effectively preserve the details feature of the image [4].

   a) b) c) d)

   Fig.1: a) RGB image b) Grayscale c) Noise removal image d) Edge enhancement image

3. LICENSE PLATE DETECTION

   1. Morphological Technique

      The basic step in recognition of license plate is to detect the plate size. In general the license plates are rectangular plate. Here, morphology [3] is used to extract the license plate.

      Preprocessing

      Edge enhancement

      Noise removal

      RGB to Gray

      I/P image

      image is divided into × blocks and then each block is processed with the binary method [5].

      2) Extraction of license plate: After the license plate is processed with the binary method, the system steps into license plate detection.

      Morphological erosion operation with the line structuring element (SE) is performed on the binary image. Erosion is a morphological transformation that can be obtained by dilating the complement of the black pixels and the taking the complement of the resulting point set. Let B denote an SE which is a line and length is 50. The erosion of the image A by SE B is the binary image z defined as follows:

      = {|() } (2)

      In erosion, every object pixel that is touching a background pixel is changed into a background pixel. It makes the object smaller. By subtracting the original image from the eroded image, the number plate would be extracted as there as a large difference between the images.

      Reconstruction is a morphological transformation involving two images and a structuring element one image, the marker is the starting point for the transformation. The other image, the mask, constrains the transformation the structuring element used defines connectivity. A hole may be defined as a background region surrounded by a connected border of foreground pixels. Fig.3 shows the result of binary, filling all holes of image and this is the final actual extracted license plate.

      a) b) c)

      Fig.3: a) Binary image b) Filling holes c) Extracted LP

   2. Histogram Based Approach

      Fig.4 shows the block diagram of LPD using histogram based approach.

      Horizontal edge processing

      Dilation

      RGB to Gray

      I/P image

      Plate region extraction

      O/P image

      Fig.2: Block diagram of LPD

      1. Binarization: The brightness distribution of various positions on a license plate image may vary due to the condition of the plate and the effect lightning enviornment. Since a binary method with global threshold cannot always generate satisfactory results in such case, the adaptive local binary method is used. The local binary method means the

         Region of interest extraction

         Fig.4: Block diagram of LPD using Histogram based

         e.

         First input RGB image converts into grayscale imag The sample of input image and grayscale image are as shown in below.

         Segmentation

         Vertical edge processing

         O/P image

         a) b) c)

         Fig.5: a) RGB image b) Grayscale image c) Dilated image

         1. Dilate an image: Dilation is the process of improvising given image by filling holes in an image, sharpen the edges of objects in an image, and join the broken lines and increase the brightness of an image. Using dilation, the noise within an image can also be removed. By making the edges sharper, the difference of gray value between neighboring pixels at the edge of an object can be increased. This enhances the edge detection. In LPD the image of a car plate may not always contain the same brightness and shades. Therefore, the given image has to convert from RGB to gray form. However, during this conversion, certain important parameters like difference in color, lighter edges of object, etc, may get lost. The process of dilation will help to remove such losses.

         2. Horizontal and vertical edge processing: Histogram is a graph representing the values of a variable quantity over a given range. In this LPD algorithm used horizontal and vertical histogram, which represents the column-wise row- wise histogram. These histograms represent the sum of differences of gray values between neighboring pixels of an image, column-wise and row-wise. In the above step, first the horizontal histogram is calculated. To find a horizontal histogram, the algorithm traverses through each column of an image. In each column, it starts with the second pixel from the top. The difference between second and first pixel is calculated. If the difference exceeds certain threshold, then it is added to total sum of differences. Then, algorithm move downwards to calculate the difference between the third and second pixel, it moves until the end of a column and calculate the total sum of differences between neighboring pixels. At the end, an array containing the column-wise sum is created. The same process is carried out to find the vertical histogram. In this case, rows are processed instead of columns [9].

         3. Passing Histogram through a Low Pass Digital Filter: It shows that the histogram values changes drastically between consecutive columns and rows in Fig.6 a). Therefore, to prevent loss of important information in upcoming step, it is advisable to smooth out such drastic changes in values of histogram. For the same, the histogram is passed through a low pass digital filter. While performing this step, each histogram value is averaged out considering the values on it right-hand side and left-hand side. This step is performed on both the horizontal histogram as well as vertical histogram. Fig.6 a) and b) show the histogram before passing though a low pass digital filter and after passing through a low pass digital filter.

         4. Filtering out Unwanted Regions in an Image: Once the histograms are passed through a low pass digital filter, a filter is applied to remove unwanted areas from an image. In this case, the unwanted areas are the rows and columns with low histogram values are removed. A low histogram values

            indicates that the part of image contains very little variations among neighboring pixels. Since, a region with a license plate contains a plain background with alphanumeric characters in it, the difference in the neighboring pixels, especially at the edges of characters number plate, will be very high. This results in a high histogram value for such part of an image. Therefore, a region with probable license plate has a high horizontal and vertical histogram values. Areas with less value are thus not required anymore. Such areas are removed from an image by applying dynamic threshold.

            a) b)

            Fig.6: a) Horizontal edge processing histogram b) Vertical edge processing histogram

         5. Segmentation: In this step is to find the entire region in an image that has high probability of containing a license plate. Co-ordinates of all such probable regions are stored in an array.

         6. Region of interest extraction: The output of segmentation process is all the regions that have maximum probability of containing license plate. Out of these regions, the one with the maximum histogram value is considered as the most probable candidate for license plate. All the regions are processed row-wise and column-wise to find a common region having maximum horizontal and vertical histogram value. This is the region having highest probability of containing a license plate. The segmented image and detected license plate as shown below:

      a) b)

      Fig.7: a) Segmented image b) Detected LP

   3. Mathematical Morphological:

      Mathematical morphology is used for to detect the license plate. It consists of two algorithms called dilation and erosion. These two algorithms are already discussed in (A & B). The process to perform first dilation and then erosion with the same mask is called a morphological closing operation. It smoothes the contours of objects. Morphological closing generally joins narrow breaks, fills long thin gulfs, and fills holes smaller than the structuring element.

      1. Detection of Discontinuities: Edge detection is the most common approach for detecting meaningful discontinuities in intensity values. Such discontinuities are detected by using first and second order derivatives. The first and second order derivative of choice in image processing is the Gradient and Laplacian respectively.

   Canny detector algorithm is very efficient to detect significant edges. This method is based on two criteria:

   • The location criterion: The distance between the actual position and the edge localized should be minimized.

   • The criterion of a response that integrates the different the different.

   The algorithm performs edge linking by incorporating the weak pixels that are 8-connected to the strong pixels.

   The template matching technique is a suitable technique for the recognition of single-font, not-rotated, and fixed-size characters. It is a technique to identify the segmented character by finding the small part in image that match with the template this method need character image as their template to store in the database. Template matching requires a library of a wide variation of character fonts and thicknesses [3], [10].

   A. Creating the template

   In order to create the templates for each character do the following operation: For every white pixel we insert the value 1 and for every black pixel 0. We do this for all the 50 training samples for each character and calculate the weights to get the template. Template after one sample of

   a) b)

   8 as shown in Fig.

   White pixel

   c)

   Fig.8: a) RGB image b) Grayscale image c) Detected LP

4. CHARACTER SEGMENTATION

   1. Connected Component

      The character segmentation process is act as a bridge between the license plate extraction and character recognition. There are several methods for character segmentation. Such as, template matching line scanning, aximally stable extreme region and connected component analysis (CCA) [5], [6]. Connected component analysis is a vital technique in binary image processing that scans binarized image and labels its pixels into components based on pixel connectivity (either 4-connected or, usually, 8- connected). Once all groups of pixels have been determined, each pixel is labelled with a value according to the component to which was assigned.

   2. Thresholding

   The thresholding is one of the simplest segmentation algorithms. In this, Otsus thresholding [11] method is used for automatic binarization level decision, based on the shape of the histogram. The algorithm assumes that the image is composed of two basic classes: Foreground and Background. It then computes the optimal threshold value that minimizes the weighted within class variances of these two classes. Character segmentation using thresholding as shown in Fig below.

   Fig.9: Character segmentation

5. CHARACTER RECOGNITION

   For the recognition of segmented character, numerous algorithms use statistical classifiers, computational intelligence architectures and Template matching.

   Black pixel

   0	0	0	0	0	0
   0	0	0	0	0	0
   0	0	0	0	0	0
   0	0	0	0	0	0
   0	0	0	0	0	0
   0	0	0	0	0	0
   0	0	0	0	0	0

   0	1	1	1	1	0
   0	1	0	0	1	0
   0	1	0	0	1	0
   0	1	1	1	1	0
   0	1	0	0	1	0
   0	1	0	0	1	0
   0	1	1	1	1	0

   Fig.10: a) Empty template b) Template after one sample 8

   Template matching is an effective algorithm for recognition of characters. The character image is compared with the ones in the database and the best similarity is measured. Calculate the matching score of the segmented character from the templates of the character stored algorithm. Compare the pixel values of the matrix of segmented character and the template matrix, and for every match we add 1 to the matching score and for every miss- match we decrement 1. This has done for all pixels. The match score is generated for every template and the one which gives the highest score is taken to be the recognized character. The character template that best matches the input characters are then displayed.

   Fig.11: Recognized LP number

6. EXPERIMENTAL RESULTS

   Computational time and success rate is computed for 25 different number plates using MATLAB2008a. The Table-I shows the detection rate, recognition rate and computational time of each method.

   TABLE I. COMPARISON OF THREE ALGORITHMS ON THE BASIS OF CHARACTER DETECTION, CHARACTER RECOGNITION AND COMPUTATIONAL TIME

   Sr. No.	Methods	Detection Rate	Recognition Rate	Computational Time (s)
   1	Morphological	95.5%	90.75%	1.1578
   2	Histogram Based	97.75%	–	2.7819
   3	Mathematical Morphology	96.50%	85.50%	3.5182

7. CONCLUSION

This paper has presented a comparison of different License plate recognition techniques. Detection rate and recognition rate have been calculated for all three methods. Histogram based method is found to be best in terms of detection rate (97.75%) and Morphological method in terms of recognition rate (90.75%). The performance of morphology method is better as compared to other methods in terms of computational time.

REFERENCES

1. Sourav Roy, Amitava Choudhary, Joydeep mukhejee, An Approach towards Detection of Indian Number Plate from Vehicle in International Journal of Innovative Technology and Exploring Engineering (IJITEE) ISSN: 2278-3075,vol.2, issue-4, pp. 241-244, March 2013.

2. S.-L. Chang, L.-S. Chen, Y.-C. Chung, and S.-W. Chen, Automatic license plate recognition, IEEE Trans. Intell. Transp. Syst., vol. 5, no. 1, pp. 4253, Mar. 2004.

3. C.-N. Anagnostopoulos, I. Anagnostopoulos, I. Psoroulas, V. Loumos, and E. Kayafas, License plate recognition from still images and video sequences: A survey, IEEE Trans. Intell. Transp. Syst., vol. 9, no. 3, pp. 377391, Sep. 2008.

4. A. S. Hadi Shari_ Kolour. An evaluation of license plate recognition algorithms, Int. Journal of Digital Information and Wireless Communications,(IJDIWC), pp. 281-287, 2011.

5. Gee-Sern Hsu, Application-oriented license plate recognition IEEE Trans. Veh. Tech., vol. 62, no. 2, pp. 552-561, Feb. 2013.

6. D. Llorens, A. Marzal, V. Palazon, and J. M. Vilar, Car license plates extraction and recognition based on connected components analysis and HMM decoding, in Lecture Notes on Computer Science, vol. 3522, J. S. Marques et al., Eds. New York: Springer- Verlag, pp. 571578, 2005.

7. Abbas M. Al-Ghaili Verical-edge-based car-license-plate detection method, IEEE Trans., on Veh. Tech., Vol. 62, no. 1, pp. 26-38, Jan. 2013.

8. Y. Wen, Y. Lu, J. Yan, Z. Zhou, K. von Deneen, and P. Shi, An algorithm for license plate recognition applied to intelligent transportation system, IEEE Trans. Intell. Transp. Syst., vol. 12, no. 3, pp. 830845, Mar. 2011.

9. Halina Kwasnicka and Bartosz Wawrzyniak, License plate localization and recognition in camera pictures, AI-METH 2002, November 13-15, 2002.

10. Regis C. P. Marques, Fátima N. S. Medeiros, Jilseph Lopes Silva and Cassius M.Laprano, License Vehicle Plates Localization Using Maximum Correlation, Structural, Syntactic, and Statistical Pattern Recognition Lecture Notes in Computer Science, Springer Berlin, 2004.

11. N. Otsu, A Threshold Selection Method for Gray Level Histogram, IEEE Transactions on System, Man and Cybernetics, 1979.