Automatic Number Plate Recognition Using Smart Phones

Automatic Number Plate Recognition Using Smart Phones

Department of Computer engineering, MMCOE, UoP, Pune, India.

Amita Mundhe Jayashree Otari

Abstract: This work develops a smart phone based system featuring license plate recognition. This system features the detection and recognition of license plate using smart phones. Using smart phone the image of a license plate is taken and the user gets to know whether the license plate is original, it also provides the details of the vehicle and the details of the owner of a vehicle. This system has been combined in three self developed novel methods, those are license plate extraction and segmentation using vertical and horizontal histogram, Ad-Hoc based remote motion control system, character recognition using artificial neural network. After implementation we can show the license plate detection rate and recognition rate of the Smart Phone based System are over 99% and over 98%, respectively, under various scene conditions. Especially, the execution time of license plate recognition, including license plate detection, is only about few seconds per frame on the Smart Phone-based System.

1. INTRODUCTION

   Smart phones are the latest need for every person in the world. Number plate detection and recognition is a necessary need in todays world too. So, For the detection and recognition there are various techniques available but detection of a plate with smart phones is an easy way. It saves time. It is the most effortless method by which the detection of a number plate can be done. This work develops a smart phone based system featuring license plate recognition. This system features the detection and recognition of license plate using smart phones. Using smart phone the image of a license plate is taken and the user gets to know whether the license plate is original, it also provides the details of the vehicle and the details of the owner of a vehicle. This system has been combined in three self developed novel methods, those are license plate extraction and segmentation using vertical and horizontal histogram, Ad-Hoc based remote motion control system, character recognition using artificial neural network. After implementation we can show the license plate detection rate and recognition rate of the Smart Phone based System are over 99% and over 98%, respectively, under various scene conditions. Especially, the execution time of license plate recognition, including license plate detection, is only about few seconds per frame on the Smart phone based system. The main Advantages of the proposed approach are, the vehicle which undergoes to an accident can be identified by using tracking technology without any delay the immediate medication will be provided to the accident victims in remote areas. To stop the duplication of License plates the detection of them

   is vey useful for IPS and to recognise the terrorist planning for anti-terrorism department this system is a real thing to look forward for.

   In future use of the proposed system will reduce the difficulty coming in manpower needed for the work and will reduce the time and energy.

2. BACKGROUND AND WORK

   With the ever-increasing demand of anti-terrorism and public security worldwide, the global law enforcement has severely been fighting against stolen vehicles or vehicles hung with stolen license plates so far. Because these vehicles, both automobiles and motorcycles, are most likely used for terrorist activities or criminal vehicles. In addition, these vehicles are always parking in or running from unimaginable corners. It is necessary for the global law enforcement to carry out the license plate investigation anywhere, anytime, even under exhausted manpower condition. Therefore, License Plate Recognition (LPR) functionality can satisfy the growing demand and gain the expanding attention.

   Today at present there are techniques available for the detection and recognition of license plate. First technique is Detection of a license plate using RCN(Readymade chassis number).In this system if the person wants to know any details about who owns the particular vehicle then only on the website of National Government www.vahan.nic.in we can find out these details. Such system is not reliable enough for the purpose of detecting the number plate because for vehicles met with an accident if the chassis number of a vehicle gets bent then there is no way to find and prove the real owner of a vehicle. And chassis numbers of

   vehicles are difficult to remember as they are very huge. For example KUN4070402820406. Plus only the name and vehicle class is not enough for knowing the details if the vehicle is stolen.

   The second system available for Detection of Number plate is with the use of Digital camera. With the use of this camera an image of a number plate is taken and it is then converted into binary number and then we come to know the details of who owns that particular vehicle.

   Fig:1 NRS using Digital Camera

   With the ever increasing demand of android phones or any smart phones in todays world a person looks for every possible feature available in the phone and camera is one important need in one of them so, there is a very rare possibility that anyone carries off camera with them. And this system in very time consuming.

   For the number plate recognition for knowing the details of owner, in other countries except India there is one system available which is called as Number plate recognition using sensors. In this system the authorised person for eg.Police can know with the help of their car sensors which are connected besides the light on the top that who owns the particular vehicle. This system makes the work of police easy to detect and control the criminal activities.

   But,this system is currently un-available in India as it needs a very advanced technology. There is a possibility that in coming years India may start using this technology.

3. PROPOSED SYSTEM

   In our proposed system 2 actors have been taken.

   1.Mobile 2.Server

   And the use cases are as follows:

   All the cases performed by the actor Mobile are Registration of the authorised person, Authentication, Capturing the image ie the image of a number plate and finally giving the records.

   The actor Server performs pre-processing on the number plate i.e. the vertical and horizontal histogram, Plate normalization in which the image will be cropped, Then the plate segmentation the binarization of a plate and the most important case

   i.e. the Neural network.

   Fig 2:Architecture

   As given in above architecture the proposed system work flow goes as first the user takes an image of the license plate from the smart phone and that image with the help of the wi-fi called as ad-hoc based remote motion control in our proposed system gets sent to the server which is the main server for the proposed system. The server sends the image to database and in the database the license plate details are searched. After the records are found the database replies the details back to the server and then the server sends the details back on the smart phone screen.

   This operation takes approximately a time of few sec. The license plate detection rate and recognition rate of the Smart Phone based System are over 99% and over 98%, respectively, under various scene conditions. Especially, the execution time of license plate recognition, including license plate detection, is only about few second per frame on the Smart Phone-based System.

   The System can be shown with its flow as follows. In the proposed system the algorihm for design of a flow of a system is given

   Step:1 Start

   Step:2 Plate Approaching

   Step:3 Edge Enhancement + Preprocessing

   Step:4 Licence Plate Normalisation Step:5 Licence Plate Segmentation Step:6 Character Extraction

   Step:7 Character Matching (Stolen plate database)

   Step:8 if Stolen plate

   Step:9 Stolen Plate Gets Detected

   Step:10 Else Goto Step 1

   Fig 3: Flow of a system

4. ANPR AND ANR

   The proposed system consists of 3 self developed novel methods namely

   1. Ad-hoc based remote motion control

   2. Plate extraction and segmentation

   3. Character recognition using Artificial Neural Network

      It can be explained as follows:

      1. Ad-hoc based Remote motion control: This s mainly used for the communication between the Because this work adopts the boot loader-unlocked Android-based smart phone platform, the Wi-Fi driver of the Android- based smart phone platform can be elaborately switched from Access Point mode to Ad Hoc mode through root certificate authority. At first, an Ad Hoc networking gateway on the Android-based smart phone platform is set up and started through the open-source utility, android-wifi-tether. Then, this work also changes the WiFi mode of the mobile robot to Ad Hoc mode, and makes the mobile robot connect to the Ad Hoc network whose Service Set IDentifier (SSID) is broadcasted by the Android-based smart phone platform. Finally, after the Ad-Hoc-based mobile robot registers an IP address to the Ad Hoc networking gateway on the Android-based smart phone platform successfully, the Android-based smart phone platform can request any motion control commands to steer the Ad-Hoc-based mobile robot remotely.

      2. Plate Extraction and Segmentation:

      Plate Extraction and Segmentation is the heart of our proposed system. This process of plate extraction and segmentation decides the actual result and the output is dependent on its result. In Plate extraction and Segmentation the 2 main histograms used are Horizontal histogram and vertical histogram.

      These 2 play a very important role in the process of extraction and segmentation. It can be explained as follows:

      This process is divided into three sub- processes namely

      • Pre-processing

      • Determination of plate kind

      • Object enhancement

        It can be explained in detail as follows:

        • Pre-processing:

          Pre-processing is very important for the good performance of character segmentation. The size of the plate images is an important factor for the accuracy of character segmentation. All the license plate images are normalized to 160*40 in pixel. It is called as size normalization. The experiments show that this scale is fit for character segmentation. This reduction of a scale is done using the down sampling matrix.

        • Determination of a plate kind:

          There are three kinds of Indian license plates: black characters on a yellow background, white characters on a blue background and white characters on a black background. The gray scale images are of two kinds: black characters on a

          white background and white characters on a black background. The ratios of number of white pixels to that of black pixels are quite different in these two kinds of gray scale images. So the kind of a plate image can be determined by histogram analysis. For that when the image is taken we know that the image is in 2D matrix in RGB colour. Our aim is to convert this RGB into Grey scale image that is to reduce 2D matrix into 1D,because we need to know the plate kind of our license plate. And to know this we note down the intensities of the colours and from that we come know the plate kind. For grey scale the intensity of a black colour is 0 and that of white is 255.

        • Object Enhancement:

          The quality of plate images varies much in different capture conditions. Illumination variance and noise make it difficult character segmentation. Then some image enhancement should be adopted to improve the quality of images. As we all know, the image enhancement methods of histogram equalization and gray level scaling have some side effects. They may have the noise enhanced as well. For character segmentation, only the character pixels need to be enhanced and the background pixels should be weakened at the same time. In fact, a license plate image contains about 20% character pixels. So these 20% character pixels need to be enhanced and the rest pixels need to be weakened. It is called object enhancement. The object enhancement algorithm consists of two steps. Firstly, gray level of all pixels is scaled into the range of 0 to 100 and compared with the original range 0 to 255, the character pixels and the background pixels are both weakened. Secondly, sorting all pixels by gray level in descending order and multiply the gray level of the top 20% pixels by 2.55. Then most characters pixels are enhanced while background pixels keep weakened. Fig. shows the result of object enhancement. It can be seen from Fig. that after object enhancement the contrast of peaks and valleys of the projection is more significant than the original.

          Fig 4. Original Image

          Fig 5.Object Enhanced Image

        • Use of Horizontal Histogram:

      For plate images with large rotation, it is difficult to obtain horizontal segment lines by horizontal projection analysis. However, for a single character, rotation has little effect on its horizontal projection. It is easier to analyze the horizontal projection of a single character and the horizontal segment lines. So the horizontal segmentation algorithm is as follows:

      1. Find valleys of the vertical projection and then vertically divide the plate image into many blocks. The division will not be very accurate because of the influence of frame and rivet. It can be seen in the figure below:

         Fig 6. Horizontal Histogram step 1

      2. Find the horizontal segmentation line for each block by analyzing the horizontal projection of the block. We call the horizontal segmentation line for a single block a subsection line. Reduce the noise It can be seen in figure below:

         Fig 7. Horizontal Histogram Step 2

      3. Find the midpoints the midpoints of all subsection lines to eliminate the incorrect subsection lines and combine the correct subsection lines into a whole line. It can be seen below:

         Fig 8. Horizontal Histogram step3

         Fig 9. Horizontal Histogram step3

         This method has a number of advantages. First, the incorrect subsection lines can be eliminated. For example, the horizontal segment lines of the block with rivet are often incorrect and can be eliminated. On the contrary, the linear fitting method is more sensitive to the incorrect subsection lines. Second, it is a local projection method, which can weaken the influence of background, illumination variance and the rotation of plate. Third, it avoids the rotation correction of images. In fact, rotation correction can cause distortion of image and make the character recognition more difficult. Figures above show some results of horizontal segmentation. The white lines denote the horizontal segmentation positions. There are images with rotation, background noise, illumination variance, rivet and plate frame influence in the figure. The results show that the horizontal segmentation algorithm has a good performance.

         • Vertical Histogram:

      The vertical segmentation algorithm is based on projection analysis, constrained by the prior knowledge. As we know, the size of license plate is 440*140(mm), each character is 45*90(mm), and the interval between characters is 12(mm). And there is a big interval (34mm) between the first two characters and the last five characters. This information is used as prior knowledge. And by using the prior knowledge, the segmentation becomes more accurate. The vertical segmentation algorithm consists of four steps, as follows:

      1. Find candidates for vertical segmentation lines. We assigned a candidate for each valley of the vertical projection.

      2. Estimate the size of the plate and each character by using the position information of the horizontal segmentation lines and the candidates.

         Fig 10.Vertical Histogram

      3. Estimate the position of the left and right borders of the big interval, using the prior knowledge of character size. The variance of the pixels gray level along a segmentation line should be small, because a segmentation line should be located in the interval of the plate and the pixels it crosses are background pixels with similar gray level. On the contrary, when it crosses a character, the gray level variance will be much larger. Based on this fact, the vertical segmentation lines (the left and right borders) for the big interval can be retrieved by searching around the estimated positions and finding the best segmentation lines with the minimum variance from the candidates.

      4. The other vertical segmentation lines can be located in the same way.

      Figure above shows some results of vertical segmentation. It can be seen that the vertical segmentation algorithm can exclude the influence of the space mark and the plate frame satisfactorily.

      3.Character recognition using Artificial Neural Network:

      We know that the artificial neural network is a network which works like human brain. If we take

      an example of AND gate taught us we know that how the and gate works:

      Input1	Input2	Output
      0	0	0
      0	1	0
      1	0	0
      1	1	1

      Table1: AND gate

      As given in the above AND gate table we know that when both the inputs are 1and1 that is TRUE and TRUE only then we get the output as TRUE. Above table is a standard table for AND gate, from which later we calculate values.

      Likewise, for our Artificial Neural network we know that there are neurons in 3 layers. In our proposed system our inputs are the license plate characters and out outputs are 0-9 numbers and A- Z characters, and the values are for black=1 and for blank = 0. We call our input as x1. For eg: for the letter M the matrix would be a 7*7 matrix. It will be in 2D image so after normalising the matrix value will come to 7*7=14. This is our x1. This is all for M. Then we know that there are inbuilt neurons in the layers of neural network. And their weight shall be considered as well which we will call as w1. And we shall call our output as v1. Which is calculated by x1.w1 . In our proposed system for the recognisation of a character we need output =1. But, There is a higher possibility that the value varies between 0.3-0.4 etc.. Consider the output as 0.4 or 0.3 , We calculate maximum of it which is 0.4. So the output is considered as 0.4. Error of machine=0.6. Here we need to adjust this error.

      For adjusting the error of machine we know that every neuron machine has learning rate=0.01. Neuron machine takes that learning rate from each output so the weights will be getting updated as the neuron machine will take learning rate from each output. Like this 1 iteration is done.

      The iterations should be performed till we reduce the machine error to 0.01. And when that happens out M character gets recognised.

      This is how the character recognition with artificial neural network is done.

5. CONCLUSION

   successfullywith the percent of 98% to 99% with a very less time of very few seconds.

6. REFERENCES

Google, Android Developers, [Online]. Available: http://developer.android.com/index.html

1. WowWee, API Specification for Rovio Version 1.2, [Online]. Available:

   http://www.wowwwe.com/static/support/ rovio/manuals/Rovio_API_Specifications_v1.2.pdf

2. Google, Android Developers, [Online]. Available: http://developer.android.com/index.html

3. C. Zhang, G. Liao, and Q. Qi, On PSF for lined motion blurred image, Proceedings of Chinese Control Conference, pp. 357360, Jul. 2008.

4. M. R. Banham and A. K. Katsaggelos, Spatially Adaptive Wavelet-Based Multiscale Image Restoration, IEEE Transactions on Image Processing, vol. 5, pp.619634, Apr. 1996.

5. Y. Wang, X. Huang, Y. Yan, and Y. Zhen, A New Method for Motion-Blurred Image Blind Restoration Based on Huber Markov Random Field, Proceedings of International Conference on Image and Graphics, pp. 5156, Sept. 2009.

6. S. Qi, H. Wang, and L. Wei, An Iterative Blind Deconvolution Image Restoration Algorithm Based on Adaptive Selection of Regularization Parameter, Proceedings of International Symposium on Intelligent Information Technology Application, pp.112115, Dec. 2009.

7. Tesseract OCR, Core Developers, [Online]. Available: http://code.google.com/p/tesseract-ocr/

8. Rojas, R. (1996). Neural Networks: A Systematic Introduction. Springer, Berlin.

In this system we have made use of Horizontal and vertical Histogram and hence for the character detection we have made use of an Artificial Neural network. After completing the proposed system we receive the result ie the plate details