Vehicle License Plate Detection From Video Using Edge Detection And Morphological Operators

Vehicle License Plate Detection From Video Using Edge Detection And Morphological Operators

Abstract – Road discipline is must due to growing number of vehicles and traffic violators on roads. In order to enhance discipline upon roads, this paper describes an instant and real time mobile vehicle license plate recognition system in an open environment. License plate recognition system plays a major role in monitoring traffic rules and maintaining law enforcement on public roads. Using a fixed camera installed at red lights, toll plaza, border crossing, and petrol station and in parking areas, system capture the videos of the vehicles. Using canny edge detection algorithm and morphological operators, system detects vehicle number plate. Experiment shows that system can effectively capture the vehicle video and detect the license plate whether it has been taken at day time, night time or under complicated environments and show accuracy up to 90%.

Keywords: LPR (License Plate Recognition), morphological operators, canny edge detector, video.

1. Introduction

   Vehicle License plate identification is an essential stage in intelligent traffic system. It is a video processing technology used to identify vehicles by their license plates. Real time LPR (License plate recognition) play a major role in automatic monitoring of traffic rules and maintaining law enforcement on public roads. In previous work, there are different techniques such as sobel edge detector [2], canny edge detection and morphological operators [1] [10], skew correction [11], color model [5] are used to detect vehicle number plate. One similarity between techniques discussed above is that they are implemented on single image and at day time only. While this paper work uses canny edge detection and morphological operators to detect vehicle number plate from the videos of the vehicles in different illumination environments. Canny edge detector [1] is an edge detection operator that uses a multi-stage algorithm to detect wide range of edges in images. Mathematical morphology [1] [10] is used as a tools to extract image

   components that are useful in representation and description of the object shape. It is mathematical in the sense that analysis is based upon set theory, topology, lattice algebra, function etc.

2. Present Work

   The central idea of this paper is to detect vehicle number plate from video. The proposed system has four modules: In first module, camera captures video of the vehicle. In second module, video get converted into frames by using MATLAB operations. In third module, frames are converted into images. Finally, in last module by using canny edge detection and morphological operators vehicle number plate is extracted. The main advantage of this technique is that from 10 sec (24fps) video, 240 frames or images are extracted. Therefore, algorithms are implemented upon 240 images one by one automatically by the system. So, this work needs more computation as compared to previous work.

   Capture Vehicle Video

   Video into frame conversion

   Frame into image conversion

   License Plate extracted

   Figure 1.1: Block diagram of License Plate Detection

   Step 1

   Capture Vehicle Video

   To capture video of the vehicle, camera must be placed at fixed location, at a minimum height of 3 feet, so that camera faces vehicle number plate. The camera takes the video from either front or rear of the vehicles. The video of the vehicle contains the license plate. The camera feeds the input video to the system. Camera can be installed at different locations. For example: toll plaza, traffic lights, border crossings, and petrol station etc. However, in this method, work has been done upon 10 to 15 sec video clip. The 10 sec video contains 240 frames/images (24fps), operations has been performed upon 240 images in order to extract vehicle number plate. This technique provides live monitoring and continuous reading. MATLAB operations are used to read the video of vehicle taken by camera positioned at fix location. Mmreader is able to read AVI, WMV, MPEG, ASF files. It creates a multimedia object for reading video files. Obj = mmreader(filename) constructs a multimedia reader object, obj (object), that can read video data from a multimedia file.

   Step 2

   Video into frame conversion

   After reading video file, convert it into frames. Frame rate taken is 24 fps. This is universally accepted film frame rate. There are 3 types of frames: I-frame, P-frame, B-frame. I-frame is an intra-frame, coded completely by itself. P-frame depend on the last P-frame or I- frame (whichever is closer) also called Anchor frame. Therefore, P-frame is constructed by predicting the difference between the current frame and closest preceding I-frame or P- frame. A B-frame is constructed from the two closest I-frames or P- frames. The B-frame must be positioned between these I-frames or P-frames. Camera captures video of different vehicles continuously and later on extracts video of single vehicle. MATLAB provides a function to extract the frames from video and process them.

   Step 3

   Frame into image conversion

   Frames can be obtained from a video and converted into images. To convert a video frames into images; the MATLAB function frame2im is used. The necessity of this conversion arises because in order to detect vehicle number plate different algorithms must be applied, these algorithms can only be applied upon images. One 10 sec video may

   produce 240 images. These images are stored in any folder, in which the further processing will be done in order to detect vehicle number plate.

   Step 4

   Number plate extraction

   In order to extract vehicle number plate, image processing has been done by using different algorithms. First of all canny edge detection algorithm is applied in order to detect edges of the vehicle images. Then, morphological operators are applied to detect the vehicle number plate area. A morphological operator is applied to the image by specifying the plate location. A morphological operator is selected that is sensitive to a specific shape in the input images. In this system rectangular box is employed as a structural element to detect the car plates.

   Mathematical morphology – It is a topological and geometrical based approach for image analysis which provides powerful tools for extracting geometrical structures and representing shapes in many applications. Using two basic operations of morphology (erosion and dilation), opening and closing of images has been done. The process of dilation followed by erosion is called closing. It has the effect of filling small and thin holes, connecting nearby objects and smoothing the boundaries contours of the objects. In closing operation dilation followed by erosion with the same SE (Structuring element). This operation removes black holes on white objects. The process of erosion followed by dilation is called opening. It has the effect of eliminating small and thin points and smoothing boundaries/contours of the objects. This operation can erae white holes on dark objects or can remove small white objects in a dark background. So, rectangular box whose size is same as number plate area will be taken as a structuring element to detect number plate area. An object will be erased if the SE does not fit within it. Hence, only vehicle number plate is detected and remaining vehicle is erased.

   1. Dilation

      (A B) (x,y) = max {A(x-s, y-t) + B(s,t)}

   2. Erosion

      (A B)(x,y) = min {A(x+s, y+t) B(s,t)}

   3. Opening

      A o B = (A B) B

   4. Closing

      A B= (A B) B

      Here A(x,y) is the gray scale image and B(x,y) is the structuring element.

      Structuring Elements- In mathematical morphology these are represented as matrices. Structuring element is a characteristic of certain structure and features to measure the shape of an image and is used to carry out other image processing operations. The shape and size of the structuring element plays crucial role in extracting vehicle number plate. 3*5 structuring element is shown below used to detect vehicle number plate.

      Figure 1.2: Structuring Element

      These algorithms are applied on 240 images one by one automatically by the system. Then system shows extracted number plates results of all images one by one automatically. Hence this is main advantage of the system that it works upon number of images not on a single one, as it is quite easy to work upon single image.

      Probability of detection of number plate from Video over single image:

      When any camera positioned at red lights captures the images of the vehicles then following problems may be faced:

      1. Probability of image accuracy is less.

      2. Corrupted or blurred image.

      3. Some movements of vehicles are missed by the camera.

         Hence, any software may be unable to detect vehicle number plate if image is inaccurate, corrupted or blurred.

         So, to overcome above mentioned problems, the need of video processing arises.

         Advantages of video processing are:

         1. Probability of getting image accuracy is more as video camera may not miss even a single movement.

         2. If one image does not show result, then there are number of images upon which operators have been applied to get the desired result.

         For example: In 10 sec, video camera produces 240 frames/images, while photo camera captures approximately 6 images.

         Hence, Probability of detection of number plate from video over images (PV) is:

         PV= ((V-I) ÷ V) * 100

         PV = ((240-6) ÷ 240) * 100 = 97.5%

         V: denotes images extracted with video camera. I: denotes images extracted with photo camera.

         In video, among 240 images approximately 218 images can show the accurate number plate. In case of photo camera, among 6 images approximately 2 images can give accurate results.

3. Experimental Results

   Vehicle Video

   Figure 1.3: Video of car

   Successfully extracted number plates from video

   Figure 1.4: Extracted number plates from video

   Vehicle number plate

   Figure 1.5: Vehicle number plate

   Table 1 shows Experimental environment, vehicles to be observed, video clips taken in different environments and length of video.

   Table 1.1: Experimental videos

   Experiment environment	Vehicles used	Experiment videos	Length of video
   Day	Car	20 clips	5 to 15 sec
   Night	Motorbike	10 clips	5 to 15 sec

   Accuracy of the above experimental videos is calculated in Table 2 shown below:

   Formula for calculating Accuracy is: Accuracy= (S ÷ I1)*100

   Here,

   S: denotes the probability of finding number plate from the video clip.

   I1: denotes number of images per video, e.g. 15 sec video produces 360 frames/images.

   Table 1.2: Accuracy calculation of vehicle number plate

   Frames per video (approx.)	Images per video (approx.) (I1)	No. Plate extracted (S)	Accuracy= (S/I1)* 100
   240	240	218	90.8%
   200	200	180	90.0%

4. Conclusion and Future work

This technique is very efficient in detecting vehicle number plate from videos of the vehicles using edge detector and morphological operators. It is highly effective in security areas like border crossing, red lights, toll plaza and petrol station etc. It shows number plate extracted from all images one by one automatically. Accuracy of showing vehicle number plate from number of images in a single video is 85% to 90%. While getting desired result of number plate from overall video is 99%. For example: among 240 images there is a single image present which shows the correct and accurate result. Future work of this technique is to select single result automatically which is best one among multiple number plates. Hence, to successfully extract single number plate from multiple number plates is the challenging future scope.

References

1. Chetan Sharma, Amandeep Kaur, Indian vehicle license plate extraction and segmentation, IJCSE, International Journal of Computer Science and Communication, Dept.of CSE, Patiala, Jul-Dec 2011, Vol 2 no.2, 593-599.

2. Dr. P.K.Suri, Dr. Ekta walia, Dr. Amit verma, Vehicle number plate detection using sobel edge detection technique, UCST, Vol 1, Issue 2, Dec 2010, 178-182.

3. Ehsan Nadersinajeb, Sara Sharifzada , Hamid Hassanpur, edge detection techniques evaluations and comparisons, IJCST, 2008, 1507-1520.

4. Kaushik Deb, Andreyvevilin,Projection and least square fitting with perpendicular offsets based vehicle license plate tilt correction SICE Annual Conference, 2010, 3291-3298.

5. Kaushik Deb and Kang Hyun Jo,Segmenting the License plate region using color model, University of Ulsan south Korea, 2009, 401-418.

6. Kou-Ming Hung, Ching-Tang Hseih,A Real- time Mobile Vehicle License Plate Detection and Recognition Tam Kang Journal of Science and Engineering, 2010, 433- 442.

7. Raphael Labayrade and Didier Aubert,In-Vehicle obstacles detection and characterization by stereovision ESV 2003 Proceedings, Nagoya, Japan, 19-22 May 2003.

8. R.C.Gonzalez and R.E.Woods, Digital Image Processing, 2nd ed. Prentice Hall, 2002.

9. Rong Lia, Musa Yassin Fortb, and Georgios C. Anagnostopoulos,Multi-stage Automatic License Plate Location and Recognition, IEEE International Conference on Image Processing, 2002, 502-511.

10. Shriram K.Vasudevan, Dharmendra T, Sivaraman R, Automotive Image Processing Technique using Cannys Edge Detector IJEST, 2010, 2632-2644.

11. Varun Manchikalapudi, Skew Correction and Localisation of Number Plate Using Hough Transform UCST, 2011, 472-476.

12. Wing Teng Ho, Wooi Hen Yap, Yong Haur Tay, Learning based License Plate Detection on Edge Features, IEEE International Conference on Image Processing, Sept.2004, 900-903.