A Comparative Study of Four Deep Neural Networks for Automatic License Number Plate Recognition System

A Comparative Study of Four Deep Neural Networks for Automatic License Number Plate Recognition System

Ganga Krishnan. G

Department of Computer Application Thangal Kunju Musaliar College of Engineering.

Kollam -691005, Kerala, India.

Natheera Beevi M

Department of Computer Application Thangal Kunju Musaliar College of Engineering.

Kollam -691005, Kerala, India.

Abstract Numerous aspects of daily life are still being transformed by technologies and services that geared towards intelligent transportation systems and smart automobiles. Automatic Number Plate Recognition has ingrained itself in our culture and is here to stay. The approach used to examine a vehicle's license plate in a photo or video collection is referred to as Automatic License Plate Recognition (ALPR) or Automatic Number Plate Recognition (ANPR). Intelligent Transportation Systems are made possible by ANPR technology, which also reduces the need for human interaction. This project aims to find out the best algorithm for license plate detection. The project uses four deep neural networks such as CNN, VGG16, VGG19, and YOLOV3 to detect the license number plate and evaluate the performance of the models in terms of accuracy and find out the best model.

KeywordsANPR, ALPR, CNN, VGG16, VGG19, YOLOV3

1. INTRODUCTION

   Automatic license plate recognition is a method for reading a vehicle's license plates in a series of images or videos. Automatic license plate recognition (ALPR) systems are getting increased attention as a result of their application in intelligent transportation systems that are installed in a number of nations for tasks including traffic law enforcement and vehicle tracking. Aside from controlling entry and exit in parking areas, gathering toll money, and keeping an eye on security in prohibited places like army campsites and guarded sanctuaries, ALPR systems can also be employed.

   These ALPR technologies are widely applied to strengthen security in specific places and prevent fraud. This activity involves a lot of labour, time, and resources, barring ALPR solutions. Additionally, it is extremely challenging for a person to precisely recall or recognize a moving vehicle's license plate, and physical interference in such activities may lead to inaccurate interpretations. If a vehicle is present in the frame of an image or video stream that an ALPR system receives as input, it displays the information on the license plate, often as text. These gadgets contain a camera for taking images of the cars. Depending on the system's parameters, the images could be in colour, black

   & white, or infrared.

   According to the fundamental definition, a number plate is "a metallic or plastic plate placed to a vehicle that helps to accurately distinguish them". However, a machine is

   unable to comprehend this definition. A machine- understandable definition is necessary to identify a number plate. A license plate may be characterized as "a rectangular region of a vehicle with a greater volume of horizontal and vertical edges" based on its characteristics. Numerous methods have been proposed to solve the license plate detection problem based on these features. While some of these algorithms are built using deep learning, others are built using older, more conventional computer vision methods. Before character segmentation and recognition, a number of pre-processing procedures are carried out in order to address the special difficulties of license plate recognition.

   Rapid urbanization is a key development in our contemporary environment. People leave rural areas and primarily choose to live in cities. As traffic in these places increases, local governments frequently overlook the mobility demands of residents and visitors, both now and in the future. A significant amount of traffic flow analysis using ANPR is being done to support intelligent transportation.

   An enormous number of moving cars may be detected and scanned using ANPR technology, which can be integrated into many aspects of the contemporary digital landscape. Although ANPR technology comes in a variety of shapes and sizes, they all serve the same fundamental purpose, which is to offer a very precise way to scan a vehicle without human intervention. Access control, smart parking, toll collection, user invoicing, tracking deliveries, traffic management, policing and security services, customer support and advice, the enforcement of red lights and lanes, queue length estimation, and many other services can all be performed with ANPR technology.

   Number plate recognition with a camera involves gathering pictures of number plates from the target scene. By processing still photographs or a photographic video using a variety of image processing-based recognition techniques, it is possible to convert the captured images into an alpha- numeric text entry. After obtaining a crisp image of the surrounding area or the target car, any ANPR system should concentrate on how trustworthy its algorithms are. The proper operation of ANPR and other smart car technologies depends on a number of key algorithms.

   Character recognition (CR) is a procedure that an ANPR system typically goes through after picture acquisition (the system's input), number plate identification [12], [13], and

   image pre-processing as output from the system [14]. Since the license plate detection and recognition stages can be clearly interpreted, many studies have used these stages to gauge performance. Another reason is that the majority of deep learning and machine learning techniques are trained using losses that are specified for each of these steps [17]. The information may be accessed once the vehicle has been successfully identified and utilized for any required post- processing operations.

   Depending on the type of camera used, its resolution, lighting/illumination aids, the mounting location, and other environment and device constraints, the image captured from the scene may encounter some complexity. The present limits lead to computationally challenging and time-consuming ALPR solving techniques [6], [7], [8], [9], [10], [11]. Due to a variety of problems, including the diversity in plate character viewpoint, shape, and format as well as varying lighting conditions at the time of photo collection, the system has had trouble segmenting and recognizing the characters [15]. Convolution Neural Networks (CNN) is a technique with excellent performance that has been employed recently. However, the CNN architecture's max-pooling layers are prone to information loss when feature maps are down-sampled [16].

   In many other systems where authorization is essential, such as smart parking systems, toll payment processing systems, etc., ALPR also offers a vital role. By automating the procedure, security officials can significantly reduce their workload. Computer vision technology has made significant advancements on a number of practical challenges in recent decades. Previously, car number plates could be recognized using template matching methods. Nowadays, number plate recognition uses numerous deep learning models that have been trained over vast amounts of data.

   The primary causes of traffic congestion and violations are the exponentially rising number of vehicles on the road. The Automatic Number Plate Recognition system's goals are to automate traffic control and decrease traffic offenses. Diverse tactics are employed in India, however, they are not particularly successful. The use of machine learning techniques to track vehicles and license plates is alreadypossible. The difficulty of these algorithms' real-time background processing, however, causes them to fail in real-time. Therefore, it is urgently necessary to create an automated system that would assist in tracking the automobiles by accurately tracing their license plates.

   The massive need for technology for traffic monitoring and management is sparked by the sharp increase in vehicle traffic on the roads. It is quite impossible to manually follow moving cars on the road in this circumstance. Losses in time and labour will occur. Even if it is operated manually, that will show incredibly tough and incredibly mistaken operations. Four neural networks were used to train the system, and its performance was assessed in order to determine the optimal approach. Traffic control, vehicle identification, and location tracking are all manual processes that increase time, expense, and effort. The introduction of ANPR technology helped to solve this issue. The suggested approach aims to boost and improve

   efficiency. Also, to find the best algorithm among different deep learning neural networks to support intelligent transportation systems.

2. RELATED WORKS

   1. Methodology for an automatic license plate recognition system using Convolutional Neural Networks for a Peruvian case study

      A system for speeding up car registration in Peru was created by Miluska Valdeos et al. [1], however it requires accurate extraction and location identification of the license plate. Here, the OpenCV library and the Python programming language are utilized. A neural network that has been trained to locate the location of the license plate is also used by YoloV4 to facilitate the deployment of an optical character recognition system (OCR).

   2. Research and Design of Automatic License Plate Recognition System Based on Android Platform

      In order to enable the personnel to use the mobile phone to swiftly and precisely gather vehicle information to complete the vehicle inspection, Pengjie Huang et al. [2] suggested an ANPR system. A system built on the Android platform and utilizing the deep neural network framework TensorFlow and the open-source computer vision library OpenCV is created. The system successfully completes the duties of locating the location of the vehicle's license plate as well as segmenting and identifying the characters on the plate. The outcomes of the experiments demonstrate that this technique is somewhat adaptable and has a positive impact on license plate recognition.

   3. Improving Robustness of License Plates Automatic Recognition in Natural Scenes

      For the recognition of license plate characters, Xudong Fan et al. [3] presented a segmentation-free network (CNNG) and a reliable license plate detection network (CA- CenterNet). No matter how the license plates are rotated or distorted, CA-CenterNet can identify not only the center of each plate, but also four vectors pointing to each of its four corners. This ability allows us to correct the distorted license plates in the source photos. The characters in the detected license plates can thus be reliably identified using CNNG without character segmentation.

   4. EILPR: Toward End-to-End Irregular License Plate Recognition Based on Automatic Perspective Alignment

      A plan for managing the cars in parking lots was put forth by Hui Xu et al. [4]. The Flask web framework and the Python programming language were used to create the web-based system. The camera's photographs were downloaded using OpenCV, and Open ALPR was used to find and examine license plates. The system runs on Linux and Raspbian operating systems. The experiments show that the technology can be applied to parking cars and can identify 85% of camera-captured photos.

   5. Automatic License Plate Detection using KNN and Convolutional Neural Network

   SegNet, a segmentation technique, is employed in a system created by P Lavanya Kumari et al. [4] to perform LPL. Semantic pixel-by-pixel segmentation is accomplished using Deep Convolution Neural Network (CNN) architecture. Utilizing adaptive contrast enhancement and Gaussian filtering, the input image is pre-processed. The LP region is located and segmented using a semantic segmentation network (SSN). For segmentation, a deep encoder-decoder network is employed. A classification is given to the segmented LP zones by CNN.

3. PROPOSED SYSTEM

   The Proposed system develops an Automatic License Number Plate Recognition System using four deep neural networks. The ANPR system is developed using CNN, VGG16, VGG19, and YOLOV3 algorithms. Each model is trained separately and compares the accuracies achieved by each model. Finally, the model that achieves the highest accuracy is selected for license plate detection.

4. METHODOLOGY

   4.1 System Architecture

   For the development of an automated number-plate detection system, it includes various steps-dataset collection, data pre-processing, algorithm selection, training the model, and number plate localization.

   Fig. 1 System Architecture

   1. Data Collection

      The procedure starts with gathering information. The process of collecting and examining data from different sources is known as data collection. For training, the dataset from Kaggle is used. The dataset contains 2 folders- the image folder and the annotation folder. The image folder contains different car images. It consists of 433 car images with a .png extension. The annotation folder contains annotation files for every image in the image folder. Annotation file contains details of each image file like height, width, depth, name, etc and it is in XML format.

   2. Data Pre-processing

      Data pre-processing is the process of modifying raw data to be utilized by a machine learning model. Pre-processing improves the data so that it can be processed later. Machine learning models cannot be created with real-world data since it frequently contains noise, missing values, and may be in an undesired format. The accuracy and efficiency of a machine learning model are increased through data pre- processing, which is required to clean the data and prepare it for the model. The training procedure is carried out only after the data is pre-processed. Images make up the data, hence picture pre-processing is used. Pre-processing includes operations like picture scaling and normalization. Resize modifies the dimensions of your photographs and, if desired, scales them. Annotations are proportionally changed. The possible values for each pixel are 0 to 256. A colour code is represented by each digit. The computation of big numeric values could be more difficult when utilizing a Deep Neural Network to process the image as it. We can normalize the data to fall between 0 and 1 to lessen this. The calculations will be simpler and quicker because the numbers will be tiny. With the exception of 0, the range is 255 because pixel values range from 0 to 256. The range will now be from 0 to 1 after dividing all of the values by 255.

   3. Training

      Training the images in the train folder comes next. It is aware of all the car's attributes. Simply said, training a model entails learning (deciding) appropriate values for each weight and bias from labelled samples. The consequence of a poor prediction is loss. In other words, the loss is a measure of how poorly the model predicted a single case. The loss is zero if the model's forecast is accurate; otherwise, the loss is higher. Finding a set of weights and biases that, on average, have low loss across all examples is the aim of training a model. MSE is the loss function in use here.

      The average squared loss across all examples in the collection is called the mean square error (MSE). Divide the total number of cases by the sum of the squared losses for each unique example to obtain MSE.

      For training 4 algorithms are used- Convolution Neural Network (CNN), Visual Geometry Group 16(VGG16), Visual Geometry Group19 (VGG19) and YOLO (You Only Look Once) V3. Train these 4 algorithms separately and evaluate the performance of these algorithm to compare the performance of these algorithms to find the best algorithm for number plate detection.

   4. Testing

   The programmer enters input and monitors the machine's behaviour and logic during machine learning testing. Therefore, the goal of testing is to clarify that the logic the machine learns is consistent. Even after several calls to the application, the logic shouldn't alter. Test the four models CNN, VGG16, VGG19 and YOLOV3 to evaluate the performance of the model.

5. RESULT

   The system evaluates the performance of four deep neural networks. Finding the best object detection method is the main objective of this study. i.e., a best object detection algorithm can accurately find out the license plate region so that the license plate number can be extracted successfully. The system automatically detects the license plate using four deep learning algorithms. The input image to the system is a car image and detects the number plate using the deep neural networks CNN, VGG16, VGG19, and YOLOV3 separately. Finally, evaluate the performance of the four deep neural networks in terms of accuracy to find out the best algorithm for license plate detection.

   The VGG16 achieves the highest performance while evaluating the performance of each algorithm using the test set. VGG19 exhibits the least accuracy as well. The VGG16 is the final model employed for number plate recognition. CNN shows an accuracy of 77%, VGG16 shows an accuracy of 89%, VGG19 shows an accuracy of 49%, and YOLOV3 shows an accuracy of 78%. Among these, VGG16 exhibits precise license plate recognition.

   Fig. 2 License Plate detection by CNN

   Fig. 3 Epoch V/S Accuracy Graph of CNN

   Fig. 4 License Plate Detection by VGG16

   Fig. 5 Epoch V/S Accuracy Graph of VGG16

   Fig. 6 License Plate Detection by VGG19

   Fig. 6 Epoch V/S Accuracy Graph of VGG19

   Fig 7 License Plate Detection by YOLOV3

6. CONCLUSION

   The study compares four deep neural networks in terms of accuracy to find out the best model for the ANPR system. For the study, dataset with both image and annotation files are used for training. The dataset is collected from the public repository for the study. The pre-processing module receives the dataset before training and prepares the data. For training the model, CNN, VGG16, VGG19, and YOLO V3 algorithms are used. Train these algorithms separately on the pre-processed dataset and perform an evaluation on these models. The evaluation result shows that VGG16 performs well on this dataset with an accuracy of 89.6% i.e., VGG16 outperforms all other algorithms for number plate detection.

7. FUTURE ENHANCEMENT Currently, the system shows that VGG16 achieves the highest accuracy. But it fluctuates according to the brightness of the light. VGG16 shows lowest accuracy in poor lighting conditions as compared to YOLOV3. So developing a system that switches algorithms automatically between yolov3 and vgg16 according to the light intensity would result in a more efficient ALPR system.

REFERENCES

[1] M. Valdeos, A. S. Vadillo Velazco, M. G. Perez Paredes, and

R. M. Arias Velasquez, Methodology for an automatic license plate recognition system using Convolutional Neural Networks for a Peruvian case study, IEEE Lat. Am. Trans., vol. 20, no. 6, pp. 10321039, 2022.

[2] P. Huang and W. Wang, Research and design of automatic license plate recognition system based on android platform, in 2022 IEEE 6th Information Technology and Mechatronics Engineering Conference (ITOEC), 2022.

[3] X. Fan and W. Zhao, Improving robustness of license plates automatic recognition in natural scenes, IEEE Trans. Intell. Transp. Syst., pp. 110, 2022.

[4] H. Xu, X.-D. Zhou, Z. Li, L. Liu, C. Li, and Y. Shi, EILPR:

Toward end-to-end irregular license plate recognition based on automatic perspective alignment, IEEE Trans. Intell. Transp. Syst., vol. 23, no. 3, pp. 25862595, 2022.

[5] P. L. Kumari, R. Tharuni, I. V. S. Sai Vasanth, and M. Vinay Kumar, Automatic license plate detection using KNN and convolutional neural network, in 2022 6th International Conference on Computing Methodologies and Communication (ICCMC), 2022.

[6] Y. Wen, Y. Lu, J. Yan, Z. Zhou, K. M. von Deneen, and P. Shi, An algorithm for license plate recognition applied to intelligent transportation system, IEEE Trans. Intell. Transp. Syst., vol. 12, no. 3, pp. 830845, Sep. 2011.

[7] D. Wang, Y. Tian, W. Geng, L. Zhao, and C. Gong, LPR- Net: Recognizing Chinese license plate in complex environments, Pattern Recognit. Lett., vol. 130, pp. 148156, Feb. 2020.

[8] S. Azam and M. M. Islam, Automatic license plate detection in hazardous condition, J. Vis. Commun. Image Represent., vol. 36, pp. 172186, Apr. 2016.

[9] A. Rio-Alvarez, J. de Andres-Suarez, M. Gonzalez-Rodriguez,

D. Fernandez-Lanvin, and B. L. Pérez, Effects of challenging weather and illumination on learning-based license plate detection in noncontrolled environments, Sci. Program., vol. 2019, pp. 116, Jun. 2019.

[10] Y.-T. Chen, J.-H. Chuang, W.-C. Teng, H.-H. Lin, and H.-T. Chen, Robust license plate detection in nighttime scenes using multiple intensity IR-illuminator, in Proc. IEEE Int. Symp. Ind. Electron., May 2012, pp. 893898.

[11] K. S. Raghunandan, P. Shivakumara, H. A. Jalab, R. W. Ibrahim, G. H. Kumar, U. Pal, and T. Lu, Riesz fractional based model for enhancing license plate detection and recognition, IEEE Trans. Circuits Syst. Video Technol., vol. 28, no. 9, pp. 22762288, Sep. 2018.

[12] R. Laroca, E. Severo, L. A. Zanlorensi, L. S. Oliveira, G. R. Goncalves, W. R. Schwartz, and D. Menotti, A robust real- time automatic license plate recognition based on the YOLO detector, in Proc. Int. Joint Conf. Neural Netw. (IJCNN), Jul. 2018, pp. 110.

[13] G.-S. Hsu, A. Ambikapathi, S.-L. Chung, and C.-P. Su, Robust license plate detection in the wild, in Proc. 14th IEEE Int. Conf. Adv. Video Signal Based Surveill. (AVSS), Aug. 2017, pp. 16.

[14] J. Shashirangana, H. Padmasiri, D. Meedeniya, and C. Perera, Automated license plate recognition: A survey on methods and techniques, IEEE Access, vol. 9, pp. 1120311225, 2021.

[15] F. N. M. Ariff, A. S. A. Nasir, H. Jaafar, and A. N. Zulkifli, Character segmentation for automatic vehicle license plate recognition based on fast K-means clustering, in 2020 IEEE 10th International Conference on System Engineering and Technology (ICSET), 2020.

[16] A. Mushthofa, A. Bejo, and R. Hidayat, The improvement of character recognition on ANPR algorithm using CNN method with efficient grid size reduction, in 2020 6th International Conference on Science and Technology (ICST), 2020.

[17] Z. Xu, W. Yang, A. Meng, N. Lu, H. Huang, C. Ying, and L. Huang, Towards end-to-end license plate detection and recognition: A large dataset and baseline, in Proc. Eur. Conf. Comput. Vis., in Lecture Notes in Computer Science, vol. 11217, 2018, pp. 261277.