Theft Vehicle Detection using Automatic License Plate Recognition

Theft Vehicle Detection using Automatic License Plate Recognition

Aromal A, Brinda H

Scholar (CSE), College of Engineering Perumon APJ Abdul Kalam Technological University, Kerala, India

Soumya K S

Shilpa J, Sreebala S

Scholar (CSE), College of Engineering Perumon APJ Abdul Kalam Technological University Kerala, India

Assistant Professor (CSE), College of Engineering Perumon APJ Abdul Kalam Technological University

Kerala, India

Abstract A large number of automobiles causes a variety of issues and disruptions in daily life. Vehicles contribute to traffic congestion on the road and vehicle theft in parking lots. Vehicle and transportation management is a time-consuming and laborious undertaking. If everything is done by hand, there will be a lot of mistakes and challenges. As a result, it is important to design an automatic recognition and detection system for vehicle number plates. This paper sheds some light on the issue at hand. With the YOLO V3 algorithm and Canny Edge Detection, the recognition system will automatically recognize the front number plate of automobiles. Following the detection of a license plate, the following actions are taken: 1.To take a picture of the license plate. 2. To recognize and segment characters. 3. A graphical user interface displays the recognized license plate, which is then saved in a database with the time and date for later use.

1. If a stolen vehicle is discovered, a notification stating that matches have been found is displayed, and a full report containing the vehicle and the time it appears is generated. . The technology can be utilized for security and accuracy purposes. The license plate number is displayed on the graphical user interface and saved in a database with the time and date for later use.

   KeywordsAutomatic License Plate Recognition; Theft Detection; Character Recognition; Canny Edge Detection; YOLO V3 Algorithm.

   1. INTRODUCTION

      In today's world, the number of vehicles is rapidly expanding. It is important to get from one location to another within a certain amount of time. We can see that there are a lot of vehicles around us. Vehicles are required by everyone for various purposes. Vehicles have increased dramatically in population proportion over the last two decades. However, it causes problems and difficulties in human life. It leads to issues such as heavy traffic, loud noise, and criminal activity such as vehicle theft and accidents, among other things. As a result, vehicle management and administration are critical to avoiding the aforementioned societal issues. As a result, several efforts are being made to improve the challenges in vehicle transportation. Vehicle Plate Recognition System is the most appealing study topic to the most recent researcher. A VPR system is a vehicle tracking system that identifies the vehicle so that it may be tracked down using an existing

      database. Typically, the identification system will be put at the residential area's gate, factory entrances, parking spaces, toll gates, university entrances, or other high- security buildings like defence institutes and nuclear power plants. Camera reflects the presence of a vehicle. It takes a picture of the car that will be utilised in future processing. A vehicle registration plate is a plate that is normally attached to the front or back of a vehicle and is composed of either metal or plastic. Numbers and alphabetical letters make up the vehicle plate number. The car's initial two letters serve as the state location prefix, followed by two number digits that represent the district to which the vehicle belongs. Number plate format of Indian vehicles are LLNN LLNNNN where L is letter and N is the number. Section II covers the extensive survey on the Literature.

      Section III describes the methodology of implementing the detection of theft vehicles using Automatic License Plate Recognition. Followed by the result of the implementation in Section IV. Finally the report is wrapped up with conclusion in Section V and the list of referenced papers as Section VI.

   2. LITERATURE SURVEY

      1. The Recognition And Detection System For Theft Vehicle By Number Plates

         The system detect and recognize license number of captured front view image of any vehicle by camera. It contains main three processes: plate extraction, character detection and character recognition. The stolen vehicle is detected by comparing it with database of stolen vehicle provided by the police station. Afterword, it rings an alarm and sends a SMS to the police station. It can be used in many security purposes like speed detection, detection of traffic violation, toll collection, parking system. It can also beneficial to secure areas like the gate of residential area, factory gates, parking space, toll plazas, university entrance or other high-secured building such as defense institutes and nuclear factories. Block diagram of entire system is shown in Fig 2.1

         Fig.1 Block diagram of number plate detection system in toll plaza.

         When, a vehicle appears in the toll plaza. It will be stopped by the gate and the image of that vehicle will captured by the camera. The camera sends the image of appeared vehicle to microcomputer unit. It extracts number plate and segmented characters. It also recognizes the getting information with help of MATLAB software. If appeared that the vehicle is stolen according to database

         ,the gate will not open for that vehicle. At the same time, the alarm rings and sends a message to police station as well as it also displays on LCD of toll plaza. In the contrast, if the captured image of vehicle will not match and exist in the database, the door will open and buzzer does not make sound or ringing. In this way, vehicle after vehicle successfully detection is going 24 hours on toll plaza.

      2. National Detection Of Stolen Vehicles

         This paper deals with the design and deployment of a National Detecting Stolen Vehicles Network System (NDSVNS) for a country and a police management authority. NDSVNS mainly uses the pattern recognition techniques to setup a Stolen Vehicle Identification System (SVIS), and integrates multiple heterogeneous databases of different city and country police departments. Each database functions as a stand-alone entity, but they are also connected by Police VPN. To share the valuable information, firstly Criminal Investigation Bureau (CIB) unified the data format of license plate recognition records and exchange formats on Police VPN. Secondly, some appropriate locations are selected in all cities and counties and toll stations of highways to install SVIS.

         Cameras installed on poles along major roads in all cities and counties are responsible for capturing image data of vehicle license plates and sending data to the License Plate Recognition Workstation. NDSVNS server is installed in the CIB Computer Center and provides police officers of all police agencies with capability to add, query, and modify license plate information and other functions. The License Plate Recognition Workstation is responsible for license plate pattern recognition, and comparing suspected stolen vehicle information to the Stolen Vehicle Database (SVD). If the vehicle is reported as stolen in SVD, the system will display a warning message on the screen and automatically sound the alarm to inform the police on duty. The police may proceed to arrest the thief. License plate number is recognized and compared to Stolen

         Vehicle Database in 0.2 second. The recognizable vehicle speed is from 0 to 100 KM/H.

         Fig.2 Architecture diagram of Natinal Detecting Stolen Vehicles Network System

      3. An Efficient License Plate Recognition System Using Convolution Neural Networks

         This paper propose an efficient license plate recognition system that first detects vehicles and then retrieves license plates from vehicles to reduce false positives on plate detection. Then, we apply convolution neural networks to improve the character recognition of blurred and obscure images. To accommodate the complexity of images taken from cameras in crossroads, this paper proposes to first detect vehicles and then detect plate on the vehicles. This method can avoid misidentifying traffic signs or advertisements as license plates. In this paper, YOLOv2 is used to detect vehicles. The Darknet-19 model adopted by YOLOv2 has 19 convolution layers and

         5 maxpooling layers. YOLOv2 first extracts features, reduces dimensions, and performs compression for an entire image using the 19 convolution layers and 5 maxpooling layers. The original images are reduced to 7*7 or 13*13. Then, YOLOv2 directly performs object recognition and predicts object positions on the reduced images.SVM is a supervised learning method used for classification and regression analysis. The process of character segmentation consists of several steps. First, the captured image is converted to grayscale and then binarized to eliminate noise. Then, perform a horizontal projection of the license plate image to determine the position of the characters arranged on the license plate. The upper and lower borders are removed by horizontal projection. Finally, perform a vertical projection on the license plate image to determine the position of each character and then divide it into single characters. In the final stage, a LPRCNN model to identify blurred and skewed characters. The proposed LPRCNN model is composed of two convolutional layers, two maxpooling layers, two fully connected layers, and one output layer. The output layer contains 34 neurons to correspond to the 34 plate characters.

      4. Vehicle Number Plate Recognition System For Theft Detection

      The presence of vehicle is detected using IR sensor after camera will capture image of vehicle which will be used for further processing. If vehicles are recognized manually then will be more mistakes with less efficiency and slow. If the described system will be

      implemented artificially by using machines as shown in fig 2.4

      Fig.3 Automatic Vehicle License Recognition System

      The crucial and initial step in ALPR system is to extract the characters of number plate from the vehicle image. The procedure for detecting is done in several stages. The number plate extraction is started with the horizontal and vertical edge detection techniques that are based on the characteristics of the edge displayed by the edges of the character on the vehicles number plate.

      The process of character recognition involves several steps like feature extraction and classification. Before recognition algorithm, the characters are normalized. Normalization is to refine the characters into a block containing no extra white spaces (pixels) in all the four sides of characters. Then each character is fit to equal size. Fitting technique is necessary for template matching. For comparing the characters with the database, input images must be same-sized with the database characters Template matching is a proper algorithm for recognition of characters. The character image is matched with the ones in the database and the best similarity is calculated. To measure the similarity and find the best match, a correlation function is used.

      For proper user-friendly environments various graphical user interfaces are created so that user can process step by step. Every recognized number plate is compared with database of stolen vehicle if math founds then that vehicle is declared as stolen vehicle. The data base of such vehicle maintained for security purpose so that stolen vehicle will be detected easily. After detecting stolen vehicle system will generate alarm and it will close door so that vehicle will not pass from that place. After message will be send to the traffic police for detecting theft.

      III . METHODOLOGY

      Automatic License Plate Recognition (ALPR) has been a frequent topic of research due to many practical applications. However, many of the current solutions are still not robust in real-world situations, commonly depending on many constraints. This paper presents a robust and efficient ALPR system based on the state-of- the-art YOLO object detector. The Convolutional Neural Networks (CNNs) are trained and fine-tuned for each ALPR stage so that they are robust under different conditions (e.g., variations in camera, lighting, and background).

      Specially for character segmentation and recognition, we design a two-stage approach employing simple data augmentation tricks such as inverted License Plates (LPs) and flipped characters. The proposed approach is divided

      into four subsections, one for each of the ALPR stages (i.e., vehicle and LP detection, character segmentation and character recognition) and one for temporal redundancy. Fig.3.1 illustrates the ALPR pipeline. We use specific CNNs for each ALPR stage. Thus, we can tune the parameters separately in order to improve the performance for each task. The models used are: Dark Flow, YOLO V3, NumPy, Tensor Flow, OpenCV and Keras.

      Fig.4An usual ALPR pipeline having temporal redundancy at the end

      1. License Plate Detection

         The video from the cctv footages is given as input to the license plate recognition system. The resolution of camera needs to be good so that the captured video can be further utilized for processing. Using Dark flow, we trained a YOLO (You Only Look Once) model with 1900 images of car with annotated plate. Labeling is a graphical image annotation tool. It is written in Python and uses Qt for its graphical interface. Annotations are saved as XML files in PASCAL VOC format, the format used by ImageNet. Dataset was composed of car images found online. YOLO v3 should be able to detect the vehicles and their LPs correctly in much less time. YOLO uses A anchor boxes to predict bounding boxes (we use A = 5) each with four coordinates (x, y, w, h), confidence and C class probabilities, so the number of filters is given by

         filters = (C + 5) × A

         Fig.5: Plate recognition

         Plate Detection is done in 2 stages using YOLO model and OpenCV functions. First Crop function use best prediction from YOLO model and return the license plate. Second Crop Function use OpenCV functions to crop a little more of the plate to avoid noise in the background.

         Fig.6: Sample dataset image

      2. Character Segmentation

         Thresholding method is used to convert the grey scale image

         to monochrome. This method reduces the complexity of captured image (input). Once the LP has been detected, we employ the CNN for character segmentation and recognition. However, instead of performing both stages at the same time through an architecture with 35 classes (0-9, A-Z, where the letter O is detected jointly with the digit 0), we chose to first use a network to segment the characters and then another two to recognize them. The character segmentation CNN is trained using the LP patch (with a margin) and the characters coordinates as inputs. As in the previous stage, this margin is defined based on the validation set to ensure that all characters are completely within its predicted LP. YOLO model trained with images of license plates where characters have been annotated. There is only one label character around 14000 characters. OpenCV is used for image processing such as image cropping etc.

         Fig.7: Segmentation by converting to grayscale

      3. Character Recognition

      It is employed for the purpose of conversion of images

      The recognized plate is compared with the number given as input. A GUI is created where the wanted number is given as input. An error message will be displayed to enter he number if number is not given as input. Only after entering the number can then the video be uploaded.

      1. RESULT

         We have used number of photos of cars by using pre-trained models. A number of images of number plates were used to correctly recognize plates. GUI was created in which the vehicle ID i.e., number plate is given as input along with which a video is uploaded for detection. Only after entering the number can then the video be uploaded. An error message will be displayed else to enter the required number. The speed of the working depends on the hardware of the system. A system with good hardware can help in the smooth working of the model. The model is trained to work with more accuracy compared to previous models.

         Fig.9: Accuracy on training

         Fig.10: GUI of model

         of

         text into characters. For Character recognition trained a convolutional neural network with Tensorflow and Keras libraries. Tensor flow is an End-to-end open-source platform for machine learning. Comprehensive, flexible ecosystem of tools, libraries and community resources that let researchers push the state-of-the-art in machine learning and developers easily build and deploy machine learning powered applications. There are 35 classes (10 for numbers and 25 for alphabet without O). Approximately 1000 images used for each classes.

         Fig8: Recognition – Data Set Sample.

         Fig.11: Error message to enter number

      2. CONCLUSION

Our proposed model is aimed to perform at higher accuracy than pre-existing models. YOLO Model and OpenCV are used for more accuracy. Vehicles can be identified automatically rather than manual ways prevailing today. Alert messages are sent immediately to the authorities. Manipulating traffic signals to help recover the wanted vehicles can be done with a few future enhancements. The proposed system of Automatic Vehicle License Plate Recognition can be implemented using above discussed method which involves fundamental Image processing steps. We can detect and recognize license number from a captured front view image of any vehicle from camera. It contains main three processes: plate detection, character segmentation and character recognition. Pre-trained models such as darkflow, keras and numpy are used. Further we can detect stolen vehicle by

comparing it with database and a message is displayed such that matches found and along with a detailed report is generated. The report indicates the image of the vehicle, the number plate and the duration in which the theft vehicle was detected. Hence the location of the theft vehicle can be traced. The proposed system can be effectively applied for the identification and detection of theft vehicles. With increased number of datasets, the accuracy of the model has been increased.

REFERENCES

1. Christos Nikolaos E Vassili Loumos, Anagnostopoulos, Ioannis

   E. Anagnostopoulos, ,and Eleftherios Kayafas A License Plate Recognition. Algorithm for Intelligent Transportation System Applications IEEE transaction on Intelligent Transportation System, Vol. 8, No. 3, Sept 2006

2. ChoudhuryA .Rahman, Wael Badawy,AhmadRadmanesh A Real Time Vehicles License PlateRecognition System Proceedings of the IEEE Conferenceon Advanced Video and Signal Based Surveillance,0-7695-1971.June 2010

3. Banshidhar Majhi Heuristics for license plate localizationand hardware implementation of Automatic License PlateRecognition (ALPR) systemDepartment of Computer Science and Engg, National Institute of TechnologyRourkela, 2012

4. Weiping Chang1 and ChingweiSu Design and Deployment of a National Detecting Stolen Vehicles Network System by 2012 Feb

5. Cheng-Hung Lin1, Yong-Sin Lin1, and Wei-Chen Liu An Efficient License Plate Recognition System Using Convolution Neural NetworksProceedings of IEEE International Conference on Applied System Innovation 2018

6. Sham Madhukar Lomte, Hajiali Sayyed The Detection and Recognition System for Theft Vehicle by Number Plates International Journal of Science and Research 2015

7. Hendry, Rung-Ching ChenAutomatic License Plate Recognition via sliding-window darknet-YOLOdeep learningDepartment of Information Management, Chaoyang University of Technology, Taichung, Taiwan Elsevier May 2019