Automatic License Plate Recognition System Based on Color Features and Vehicle tracking

Automatic License Plate Recognition System Based on Color Features and Vehicle tracking

K. Subhalakhsmi and V. Siva Soundharam

Students, MRK Institute of Technology, Kattumannarkoil, Nattarmangalam, Cuddalore, Tamil Nadu

AbstractIn this paper, an vehicle license plate recognition system based on a new localization approach, which is modified to reflect the local context, is proposed, along with a hybrid classifier that recognizes license plate characters. The method presented here is based on a modified template matching technique by the analysis of target color pixels to detect the location of a vehicles license plate. A modified strip search enables localization of the standard color-geometric template utilized iranian and several European countries. This approach uses periodic strip search to find the hue of each pixel on demand. In addition when a group of target pixels is detected, it is analyzed to verify that its shape and aspect ratio match those of the standard license plate. In addition to being scale and rotation invariant, this method avoids time-consuming image algorithms and transformations for the whole image pixels, such as resizing and Hough, Fourier, and wavelet transforms, thereby cutting down the detection response time. License plate characters are recognized by a hybrid classifier that comprises a decision tree and a support vector machine witha homogeneous fifth- degree polynomial kernel. The performance detection rate and the overall system performance achieved are 96% and 94%, respectively.

Index TermsColor template matching, image recognition, license plate detection, license plate localization, license plate number identification, license plate recognition (LPR).

1. INTRODUCTION

   Automatic license plate recognition system (ALPR) has become a high-priority research in recent years. Because the license plate is a unique ID for a vehicle, its automatic recognition has many uses. For example, the license plate recognition (LPR) system can be used in smart parking areas or smart toll stations to open gates for vehicles bearing authorized license plates or to calculate the average speed of a vehicle between two stations by recognizing its license plate at both stations. In addition, by installing LPR systems on roads, particularly in traffic zones and at junctions that need police patrolling, prohibited vehicles can be recognized and their movement monitored. Nowadays, speed control stations on highways take color photos of vehicles that break the speed limit. Usually, on highways in Iran, these control stations use local memory because they are not connected to a central database [1]. The photos are taken by speed control cameras in high resolution, which consume a lot of disk space; thus, after a while, most of the stations encounter a low-disk-space problem. This problem can be solved by the ALPR system, which converts huge data of images into a series of bits. The ALPR system installed at speed control

   stations uses high resolution images to recognize license plates. After recognition, the images are compressed into small and low-resolution images and then transferred through normal and low-band connection devices such as short messaging or multimedia messaging by a GSM board. Instead of mere recognizing and sending traffic tickets to the offenders after a few days, penalizing the offenders at the time and place of the incident would be more effective in preventing accidents and deterring drivers from repeat offenses. Thus, an ALPR system based on color images from normal/high-resolution surveillance cameras is highly useful. That is why several studies have applied color features for localizing license plates. Most of their algorithms, which apply color features for license plate localization, are country specific [2]. For localization, they usually convert all image pixels to other color spaces, such as hue, saturation, and value (HSV) or hue, saturation, and intensity (HSI), and find background or foreground color using tiling and hue histogram[3]. Some algorithms apply color features and fuzzy sets to localize license plates [4], [5]. Color edge detection is widely used for license plate detection; in addition, it has been applied as an aided feature in Persian license plate detection [2], [4],[6]. Most of the implemented LPR systems for Iranian license plates are based on infrared photos and, hence, require special photographic equipment for infrared photography [1]. The proposed system, which is based on developments in image processing and optimized template-matching methods, proves that a fast LPR system based on color features is possible and practical. It recognizes license plates in color images without any resizing and conversion, thus reducing the response time. It can be used in automatic toll stations, tunnel, highways, intelligent parking, and traffic zones for LPR by using surveillance cameras. The results based on images from speed control cameras on highways demonstrate the systems capability and reliability. The functioning of the ALPR system includes two steps. In the first step, the system performs license plate localization, which involves identifying the license plate location in the original image and cropping it. In the second step, the system extracts and classifies all the available

   Fig. 1. (a) Samples of infrared images in daylight. (b) License plate detection through horizontal projection in an image.

   license plate is close to the camera [18], [20], [30]. Many previous studies used hybrid methods to achieve a more accurate and faster license plate detection, for example, combining fuzzy logic and neural network [2], [27].

   As aforementioned, most algorithms in license plate localization ignore color features and use gray-level or binary processing to localize license plates. Using color feature for vehicle license plate localization is nearer to human vision and decision [7]. Therefore, we propose a new algorithm based on color feature that requires low- level processing to find minimum candidates.

   1. New Approach for License Plate Detection

      The proposed method for license plate localization is an optimized template-matching algorithm. It is basically an algorithm that sweeps the entire image surface to find the object via a template or its description [25], [31]. Usually, a template-matching system is not flexible and expandable enough to accommodate imported images of any size [25]. The proposed method, on the other hand, offers optimized template matching that is scale invariant and orientation invariant so that it can detect license plates of any scale or size and in any direction. For using a template-matching algorithm, a template or its description is necessary. Thus, defining a salient and standard template for Persian license plates is important. A suitable feature is the blue rectangle that appears on the side of Iranian and some European license plates. The license plate can be detected by finding the blue rectangle in terms of its standard aspect ratio. Its detection is facilitated by its color-geometric description as a mask. Fig. 2 shows the blue rectangle on license plates of Iran and some European countries. Because color information is used to detect the location of a license plate, converting the whole image into gray level, black and white, or binary level is not necessary. Detecting an object within its image has always been difficult in image processing[25]. Generally, template- matching methods sweep an image, pixel by pixel or window by window, for comparing a template with a part of the image using convolution. The part with the lowest error rate is tken as the candidate for matching. This whole process imposes a heavy overhead on the system [31]; therefore, it is necessary to find an approach that is scale and rotation

      invariant and that can sweep the entire image surface at high speed. The proposed method uses color feature to localize a license plate. The RGB channel format is a natural feature that correlates all the channels with one another for representing real-world colors. HSV is a component of the color space that includes brightness and saturation of the basic hue. Hue indicates the dominant wavelength of color, and saturation indicates its purity.

      Highly saturated colors contain a very narrow set of wavelengths that are much more pronounced than similar but Less saturated colors [31]. In the primary step of localization, blue pixel colonies are sought by using hue. Color spaces with separated hue, such as HSV and HSI, are quite useful for seeking. The proposed method seeks blue pixel colonies to find blue rectangles in a license plates aspect ratio. Converting all image pixels from RGB to HSV is not necessary, but peri are converted on demand for a vertical or a slope search using (1) and condition. Equation (1) finds bluish of a pixel; if conditions are not true, the pixel is not blue, and it is assumed as do not care. Equation (1) is shown at the bottom of the page, where R, G, and B are the three RGB matrices, respectively; is the number of columns (width), and n is the number of rows (length) in an m × n image resolution. determines the low saturation pixels (white pixels) and the low-value pixels (black pixels) that need to be disregarded. is the initial parameter that is adjustable by the user in special cases; it is 0.21 by default.odic strip line pixels

   2. Confirming License Plates

   After finding all the candidates through vertical or slope search, the candidates are cropped from the original image. However, one question remains: Which one is the correct license plate? The proposed system localizes all the available ASHTARI et al.blue rectangles in the same aspect ratio of the license plate and crops them. The projection analysis confirms the real plates in the primary step. The projection method has to include at least eight independent peaks (because a standard Iranian license plate includes seven numbers and a letter), as shown

   Fig. 2. Samples from cropped car plate after detection. (a) Original cropped license plate. (b) Applying histogram equalization and gamma adjustment. (c) Applying the Laplacian filter.

   Peaks in projection are achieved by two simple arrays and by finding semilocal maximum and minimum. If the peaks are recognized, the system could successfully find the real license plate; otherwise, the system rejects and deletes the candidate from the candidate list. Finding vehicle lights or bumper area by texture, shape, and color analysis is a useful way to detect the real license plate. Considering that a few images contain fake blue rectangles in the same shape and aspect ratio of the license plate and that, statistically, the proposed system rarely detects fake blue rectangles as a license plate (1.27%), the process of finding vehicle lights or bumper is unnecessary and time consuming.

2. RESULTS OF EXPERIMENTAL SETUP

   The performance of the proposed method has been evaluated under four different conditions: 1) the vehicle is in the speed lane [see Fig. 3(a)]; 2) the vehicle is in the middle lane [Fig. 3(b)]; 3) the vehicle is in the side lane [see Fig. 3(c)]; and 4) the image includes license plates of several vehicles [see Fig. 3(d)]. The proposed detection method is possible to detect several license plates in an image. The proposed system is considered successful in that it at least detects and recognizes a license plate correctly. A standard speed control station should cover and focus on a lane, and the speed lane is the target lane for Iranian speed control cameras. Therefore, photos of vehicles in other lanes, taken by this type of speed control camera, are not standard. Our evaluation, based on the speed lane, ensures 96.6% performance in detection rate and 95.85% performance in recognition rate. In addition, the proposed system ensures

   92.6% accuracy in the overall system performance. Detection and recognition rates under three different

   conditions of photographydaylight, straight; night light, straight; daylight, at an angle of up to 20

   1. SYSTEM RESPONSE/COMPUTATION TIME

      The assessment has been done under two different conditions: the direction of the license plate is given and primary slope sweeping is applied to each under processing colony without using (3)and (4); and the orientation of the license plate is unknown and the moving window method is applied to each under processing colony. What is shown in Table VI is the average elapsed time for 100 runs in a system with a dual-core 1.7-GHz

      Fig 3. Performance of the proposed method has been evaluated under four different conditions

3. CONCLUSION

A new approach for license plate detection, based on the color features in Iranian license plates using periodic vertical or slope sweep, has been presented. The proposed method detects the location of license plates by recognizing its hue and shape. The localization is scale and rotation invariant. The proposed localization system detects all the license plate candidates available in a scene. After detecting the license plate candidates, it extracts them and passes on the image to the recognition section for recognizing the eight characters in each Iranian license plate. The proposed detection system localizes minimum candidates and detects the real candidate (i.e., real license plate) directly in 98.73% of the images. 1704PU and 4-GB RAM. After successful detection, the quality of the license plate image is improved, particularly in the characters edges, by applying image processing filters, such as histogram equalization and the Laplacian filter. All surplus components are removed by applying some rules and morphological functions. Characters in the license plate are extracted by connected component or projection analysis and passed on to the hybrid classifier for recognition. The hybrid classifier comprises a decision tree and an SVM.

Fig 4. Graphical user interface of Iranian ALPR software

Color images for the proposed system can be generated by

normal surveillance photographic equipment; hence, professional photographic equipment, such as infrared camera, is not necessary. The types of Iranian license plates, such as public license plates with yellow background or government license plates with red background, were identified by a hue histogram in the detection section. The proposed method has been extensively analyzed; in addition, it has been completely implemented and has proven to be practical. Fig. 4 shows the graphical user interface of Iranian ALPR software based on the proposed detection and recognition methods. The main application that has been programmed via Java saves the plate number, the vehicles speed, the coordinates of the license plate location, the percentage accuracy for each recognized character, the date, the time, and the address and location of the resized original image in a database. Efforts to provide faster and more accurate ALPR systems will have to continue. A system that recognizes license plates in all the weather conditionsfoggy or rainyand even when the license plates are broken or smeared, as efficiently and reliably as does human vision, is a distinct possibility in the near future.

ACKNOWLEDGMENT

The authors would like to thank Dr. A. Amiri for his enthusiastic guidance throughout this research work. They would also like to thank the anonymous reviewers for their constructive comments.

REFERENCES

1. A. H. Ashtari, License plate localization and recognition in color images based on new feature of Iranian license plate and support vector machine, M.S. thesis, ept. Eng., Islamic Azad Univ., Dezful, Iran, 2009.

2. C. N. E. Anagnostopoulos, I. E. Anagnostopoulos, I. D. Psoroulas, V. Loumos, and E. Kayafas, License plate recognition from still images and video sequences: A survey, IEEE Trans. Intell. Transp. Syst., vol. 9, no. 3, pp. 377391, Sep. 2008.

3. W. Jia, H. Zhang, X. He, and M. Piccardi, Mean shift for accurate license plate localization, in Proc. IEEE Intell. Transp. Syst., 2005, pp. 566571.

4. S. L. Chang, L. S. Chen, Y. C. Chung, and S.W. Chen, Automatic license plate recognition, IEEE Trans. Intell. Transp. Syst., vol. 5, no. 1, pp. 42 53, Mar. 2004.

5. X. Shi,W. Zhao, and Y. Shen, Automatic license plate recognition system based on color image processing, in Proc. ICCSA, 2005, pp. 307314.

6. V. Abolghasemi and A. Ahmadyfard, An edge-based color-aided method for license plate detection, Image Vis. Comput., vol. 27, no. 8, pp. 1134 1142, Jul. 2009.

7. R. W. Rodieck, The First Steps in Seeing. Sunderland, MA, USA: Sinauer Associates, 1998.

8. E. R. Lee, P. K. Kim, and H. J. Kim, Automatic recognition of a car license plate using color image processing, in Proc. IEEE ICIP, 1994, pp. 301305.

9. C.-C. Lin and W.-H. Huang, Locating license plate based on edge features of intensity and saturation subimages, in Proc. 2nd ICICIC, 2007, p. 227.

10. F. Wang, L. Man, B. Wang, Y. Xiao, W. Pan, and X. Lu, Fuzzy- based algorithm for color recognition of license plates, Pattern Recognit. Lett., vol. 29, no. 7, pp. 10071020, May 2008.

11. Y. Yanamura, M. Goto, D. Nishiyama, M. Soga, H. Nakatani, and

    H. Saji, Extraction and tracking of the license plate using Hough transform and voted block matching, in Proc. IEEE Intell. Veh. Symp., 2003, pp. 243246.

12. T. D. Duan, T. L. H. Du, T. V. Phuoc, and N. V. Hoang, Building an automatic vehicle license plate recognition system, in Proc. Int. Conf. Comput. Sci. RIVF, 2005, pp. 5963.

13. N. Zimic, J. Ficzko, M. Mraz, and J. Virant, The fuzzy logic approach to the car number plate locating problem, in Proc. IIS, 1997, pp. 227230.

14. S. M. Youssef and S. B. AbdelRahman, RETRACTED: A smart access control using an efficient license plate location and recognition approach, Exp. Syst. Appl., vol. 34, no. 1, pp. 256 265, Jan. 2008.

15. H. Kwanicka and B. Wawrzyniak, License plate localization and recognition in camera pictures, in Proc. 3rd Symp. Methods Artif. Intell., 2002, pp. 243246.

16. H. Bai, J. Zhu, and C. Liu, A fast license plate extraction method on complex background, in Proc. IEEE Intell. Transp. Syst., 2003, pp. 985987.

17. D. Zheng, Y. Zhao, and J. Wang, An efficient method of license plate location, Pattern Recognit. Lett., vol. 26, no. 15, pp. 2431 2438, Nov. 2005.

18. H. Zhao, C. Song, and S. Zhang, License plate recognition system based on morphology and LS-SVM, in Proc. IEEE GrC, 2008, pp. 826829.

19. J. Jiao, Q. Ye, and Q. Huang, A configurable method for multi- style license plate recognition, Pattern Recognit., vol. 42, no. 3, pp. 358369, Mar. 2009.

20. S. H. Kasaei, S. M. Kasaei, and S. A. Kasaei, New morphology based method for robust Iranian car plate detection and recognition, Int. J. Comput. Theory Eng., vol. 2, no. 2, pp. 264 268, Apr. 2010.

21. F. M. Kazemi, S. Samadi, H. R. Poorreza, and M. R. Akbarzadeh-T,

    Vehicle recognition based on Fourier, wavelet and curvelet transforms A comparative study, in Proc. ITNG, 2007, pp. 939 940.

22. F. M. Kazemi, S. Samadi, H. R. Poorreza, and M. R. Akbarzadeh-T,

    Vehicle recognition using curvelet transform and SVM, in Proc. ITNG, 2007, pp. 516521.

23. V. Beëanoviéa, M. Kermitb, and A. J. Eidec, Feature extraction from photographical images using a hybrid neural network, in Proc. SPIE 9th Workshop Virtual Intell./Dyn. Neural Netw., 1999, vol. 3728, pp. 351361.

24. K. Jung, Neural network-based text location in color images,

    Pattern Recognit. Lett., vol. 22, no. 14, pp. 15031515, Dec. 2001.

25. R. Brunelli, Template Matching Techniques in Computer Vision: Theory and Practice. Hoboken, NJ, USA: Wiley, 2009.

26. M. H. Dashtban, Z. Dashtban, and H. Bevrani, A novel approach for vehicle license plate localization and recognition, Int. J. Comput. Appl., vol. 26