Quality Inspection of Tire using Deep Learning based Computer Vision

Quality Inspection of Tire using Deep Learning based Computer Vision

1 Mr. Harshal Bhamare

1 M.E. Student,

Department of Information Technology, VIT, Wadala, Mumbai, India

2 Prof. Ajitkumar Khachane

2 Asst. Professor Department of Information Technology,

VIT, Wadala, Mumbai India

Abstract This system measures the depth of tire treads using LabVIEW stereo vision and can determine the correct depth in the treads region of interested (ROI) using image processing for edge detection. In addition, this allows a user to acquire information about the depth through the Internet or a phone by connecting to a database with the Internet of Things. The role of the tire tread groove is to increase the friction between the tire tread and the road and to remove the water between the tire and the road to avoid slip- page. For safety purpose, it is demanded that the vehicles tire tread depth could not be reduced below the limiting value. When the tread depth is below 1.6 mm, the tire should be replaced. Traditionally, the tire tread depth is measured with a depth indicator when the vehicle is at rest. This kind of measurement is inefficient, human factor dependent, and inconvenient. For the purpose of increasing the efficiency and reducing labor intensity, attempts have been made to facilitate the measurement of tires tread depth with image sensors. Such devices consist of an array of laser light sources and multiple image sensors arranged in a line to acquire the tires profile. Those systems reported do not involve automatic identification of the tires grooves. However, the identification of the grooves on the tires outer most surface is necessary for automatic measurement.

1. INTRODUCTION

   Tires are one of the core components of any automobile and manufacturers need to ensure their quality through various inspections. Our Solution offers robust and reliable tire inspection systems, well suited for specific inspection requirements of automobile manufacturers. All the systems, including all supporting devices, are manufactured to meet or exceed the required automobile standards and regulations for radiation safety. If we consider some personal vehicle or fleet owner who is having a greater number of vehicles, then our proposed previously traditional approach for tire inspection is penny test. But that solution so much simple but it is not that much accurate microscopic level thats why we do complete tire inspection by capturing tire picture through mobile application. Using tire inspection, we can identify tire sidewall texture, logo and another meta information so that we can identify tire details like tire vendor detail information, tire quality, tire used for which category of vehicles or appliances. One biggest solution is tire tread depth measurement because of it we can easily identify life of tire, defect of tire whether tire is punctures, dents, cuts, scrapes, bulges, bumps or cracks. Means our proposed solution implements complete life cycle of tire. The proposed solutions have all computation that will happened in background that solution is based on

   Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) viz. we need to build CNN+RNN based deep learning model and that model need to train on lakhs of tire sidewall picture as well as tire tread pictures.

2. LITERATURE REVIEW

   We are searching various Text detection and text recognition algorithm implementation like EAST which is TensorFlow based re-implementation An Efficient and Accurate Scene Text Detector and it is tested on various competition datasets like ICDAR 2015, COCO-Text and MSRA-TD500.Given algorithm also optimized and more accurate in Advanced EAST.

   Another algorithm for text detection and text recognition: We study how we localize and detect text in natural scenes using algorithm is SynthText i.e. Synthetic Data for Text Localization in Natural Scevne Images invented by Ankush Gupta, Andrea Vedal di, Andrew Zisserman, CVPR 2016. This algorithm is written in python using python library pygame, opencv (cv2), PIL (Image), numpy, matplotlib, ppy, scipy.

   In Tire Tread Depth calculation, we study how to calculate tire tread depth using image triangularization this paper shows non-contact measurement approach to calculate tire tread depth also reduce the manpower expense and improve the convenience of measurement, that solutions based concept of machine vision, Epipolar plane, tread depth, non-contact measurement, android smart phone application. Tire sidewall embossed information can be read using method for modelling the tire sidewall surface to fetch the embossed or engraved character by concept of base-line correction and then the statistical technique is used to generate the residual data to relief the embossed surface. A prototype system of the laser-scanning for tire sidewall is implementer. In that algorithm, it uses concept of Orthogonal polynomial, 3D surface modeling, Embossed segmentation. After analysis of literature survey, I got some algorithm based on computer vision, image processing some are based on machine learning, Deep Learning, Artificial Intelligence. Above discussed IEEE Papers are based on Deep Learning Some are based on Computer Vision that are follows. Real-time Surface Acquisition of Tire Sidewall for Reading Embossed Information, Measurement of Tire Tread Depth with Image Triangulation these research papers based on Image processing which are useful for tire tread depth calculation, tire performance quality. These papers basically based on concepts and algorithms

   of computer vision, image processing. Remaining paper are useful for detecting tire texture and other details using deep learning or machine learning.

3. OBJECTIVE OF THE RESEARCH WORK Self-tire inspection through mobile application interface so that end user can predict tire life and defect detection,

   punctures, dents, cuts, scrapes, bulges, bumps or cracks in

   tire. Any tire consists of two main parts: treads, which come in contact with the road, and sidewalls that contain many information about the tire such as manufacturer name, radial, etc. The DOT code is a part of information printed on the sidewall. The DOT code as shown in Fig. 1 begins with the word "DOT" followed by codes that specify company, factory, mold, batch, and date of production. The last four digits represent the week and year when the tire was made. To manually classify different types of tires and its life, it takes a lot of time and effort

   Figure 1: DOT code region on tire sidewall image

4. PROBLEM FORMULATION

   In this system our aim is to develop a system using machine learning or deep learning approach i.e. supervised learning algorithm approach firstly we need to gather various images vendor and tire type wise then we need classify that data set according to test dataset and training dataset approach. In this problem we will explore various deep learning algorithms which are based on Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) that are R-CNN, Fast R-CNN, Faster R-CNN, YOLO, Long Short-Term Memory (LSTM) etc. From that various algorithmic approaches, we analyses which is better. More number of images gives more accuracy in result of our output which is nearer to 99.99%. In this system, we will explore various deep learning frameworks i.e. TensorFlow, keras, Caffe, pytorch, theano etc. from these frameworks which is better for developing and maintaining source code.

   1. Collection Dataset:

      In the proposed system main part is to collect the data i.e. dataset contains all images tht images are needing to be pre-processed so that following preparation points need to consider

      • Identify Outliers in your Dataset

      • Improve Model Accuracy with Data Pre- Processing

      • Discover Feature Engineering

      • An Introduction to Feature Selection

      • Tactics to Combat Imbalanced Classes in your

        Machine Learning Dataset

        • Data Leakage in machine Learning or deep learning

          We further divide our dataset in test Dataset and training Dataset. Training dataset is used for building deep learning model and test dataset is used for getting prediction of tire text recognition and tire tread measurement.

5. MOTIVATION/ PROBLEM STATEMENT DEFINITION

   Traditionally end users i.e. drivers, fleet owners did self- tire inspection using penny test.

   Once every month, or before you embark upon long road trips, check your tires for wear and damage problems. One easy way to check for wear is by using the penny test. All you have to do is grab an Abraham Lincoln penny and follow 3 easy steps.

   • Take a penny and hold Abe's body between your thumb and forefinger.

   • Select a point on your tire where tread appears the lowest and place Lincoln's head into one of the grooves.

   • If any part of Abe Lincoln's head is covered by the tread, you're driving with the legal and safe amount of tread. If your tread gets below that (approximately 2/32 of an inch), your car's ability to grip the road in adverse conditions is greatly reduced.

   The research related to this problem is still in its early stages, hence, no mature solutions are available yet, and available knowledge in this field is not coherent enough to form strong basis. The main paper that was found in this field is titled "Pattern Recognition for Classification and Matching of Car Tires". The following is a description of its work. This paper is interested in whether a given tread pattern matches an existing tire; they saw that the only way at that moment is to compare it manually and search in a large library of tire tread patterns. So, they work to automate this pattern matching using computer vision, image processing and pattern recognition.

6. PROPOSED RESEARCH METHODOLOGY In this system our aim is to develop a system using machine learning or deep learning approach i.e.

   supervised learning algorithm approach firstly we need to

   gather various images vendor and tire type wise then we need classify that data set according to test dataset and training dataset approach. In this problem we will explore various deep learning algorithms which are based on Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) that are R-CNN, Fast R-CNN,

   Faster R-CNN, YOLO, Long Short-Term Memory (LSTM) etc. From that various algorithmic approaches, we analyses which is better. More number of images gives more accuracy in result of our output which is nearer to 99.99%. In this system, we will explore various deep learning frameworks i.e. TensorFlow, keras, Caffe, pytorch, theano etc. from these frameworks which is better for developing and maintaining source code.

   6.1 Method for tire tread depth measuring with image pattern:

   In the research, a prototype tire tread depth measuring system was designed as shown in Figure 5. The system consists of two cameras, two laser emitters, a reflecting mirror, a supporting plate, a trigger switch and the soft- ware based on LabVIEW. There are two windows on the supporting plate for the laser plane passing through and the reflecting mirror seeing the tire. Two cameras (the left camera and the right camera) are fixed under the supporting plate, and the cameras principal axis is parallel to the ground and pointing to the mirror. The principal axis of the lens passes through the center of the mirror. The distance between the mirror center and the lens is 250 mm. With this distance, each camera has a view width of 250 mm with the selected focal length. Two cameras could cover a view width of 500 mm which is enough for the passenger car tire measurement. The parameters of the cameras are shown in Table 1. The mirror was placed for the cameras to see the image of the tire in the mirror. The angle between the mirror and the ground is 45°. The lasers adopted are line laser emitters, and the trigger adopted is a photoelectric switch. The parameters of the line laser emitter and those of the trigger are

   shown in Table 2.

   Laser emitter terms	Parameters	Trigger terms	Parameters
   Product name	Line laser	Product name	The photoelectric switch
   Product model	LT-B6305- GLD	Product model	LX12- DJ30MNK
   Light form	Line	Detection distance	030 m
   Line width	0.5 mm	Inductive approach	The laser correlation type
   The opening angle of light	10°110°	The output form	NPN normally open
   Working voltage	DC3-6 V	Working voltage	DC6-36 V

   Laser emitter terms	Parameters	Trigger terms	Parameters
   Product name	Line laser	Product name	The photoelectric switch
   Product model	LT-B6305- GLD	Product model	LX12- DJ30MNK
   Light form	Line	Detection distance	030 m
   Line width	0.5 mm	Inductive approach	The laser correlation type
   The opening angle of light	10°110°	The output form	NPN normally open
   Working voltage	DC3-6 V	Working voltage	DC6-36 V

   Table 1. Camera parameters.

   Table 2. Laser emitter and trigger parameters.

   Items	Left camera	Right camera
   Sensor type	CCD	CCD
   Sensor size	4.9 mm 3 3.6 mm	4.9 mm 3 3.6 mm
   Resolution (H 3 V)	1294 pixels 3 964 pixels	1294 pixels 3 964 pixels
   Pixel size (H 3 V)	3.75 mm 3 3.75 mm	3.75 mm 3 3.75 mm
   Frame rate	30 fps	30 fps
   Mono/color	Mono	Mono
   Focal length	fx = 6.1 mm, fy = 6.1 mm	fx = 6.3 mm, fy = 6.2 mm
   Optical center	u = 630.8 pixel, v = 491.4 pixel	u = 632.6 pixel, v = 452.1 pixel

   The main reason for the application of the line laser emitter is that the laser forms a laser plane and its projection is a line strip. The laser line width is 0.5 mm and the maximum opening angle of the laser emitter is 110°. Two-line laser emitters can completely cover the tire tread surface, and two laser emitters were installed to form one laser plane. The laser plane is placed in front of the mirror, and the angle between the laser plane and the ground is 45°. The main reason that a photoelectric switch was applied is that the photoelectric switchs trigger position on the supporting plate is a lin.

7. POSSIBLE OUTCOME / RESULT

   In just a few years, deep learning almost subverts the thinking of image classification, speech recognition and many other fields, and are forming an end-to-end model in which the most reprehensive deep features can be learnt and classified automatically. This model tends to make everything easier. Moreover, in deep nets each layer can be adjusted according to the final task and ultimately to achieve co-operation between the layers which can greatly improve the accuracy of the task.

   Fig: snapshot of outcomes

8. CONCLUSIONS

   The proposed system will target to various tire making vendors, personal vehicle users i.e. drivers, fleet owners. Automatic quality inspection is strongly desired by tire industry to take the place of the manual inspection. Different from the existing tire defect detection algorithms that fail to work for tire tread images, the proposed

   detection algorithm works well not only for sidewall images but also for tread images.

   We will be developing complete tire life- cycle system. The Solutions indicated that the correct tire tread depth could be obtained from seven of the eight images of the same tire. In other words, valid tire tread depth images could be acquired from the depth images. Furthermore, the correct tire tread depth could be analyzed from these images, which could then be uploaded to the android phones of a user through the No-SQL database.

9. REFERENCES

1. S. Sukprasertchai, T. Suesut, Real-time Surface Acquisition of Tire Sidewall for Reading Embossed Information, International Multi Conference of Engineers and Computer Scientists 2016 Vol I, IMECS 2016, March 16 – 18, 2016, Hong Kong.

2. Shih-Yen Huang, Yen – Cheng Chen, Jih-Kai Wang, Measurement of Tire Tread Depth with Image Triangulation, 2016 International Symposium on Computer, Consumer and Control.

3. Yan Zhang1, Xuehong Cui, Yun Liu, Bin Yu, Tire Defects Classification Using Convolution Architecture for Fast Feature Embedding, International Journal of Computational Intelligence Systems, Vol. 11 (2018) 1056-1066, May18, 2018.

4. Oskar Svensson, Simon Thelin, Indirect Tire Monitoring System – Machine Learning Approach, Degree Project in Computer Engineering, Helsingborg, June 26, 2017.

5. Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, Jiajun Liang, EAST: An Efficient and Accurate Scene Text Detector, Megvii Technology Inc., Beijing, China, July 10, 2017.

6. Gupta, A. and Vedaldi, A. and Zisserman, A. Synthetic Data for Text Localisation in Natural Images, IEEE Conference on Computer Vision and Pattern Recognition, 2016.

7. Ankush Gupta, Andrea Vedaldi, Andrew Zisserman, "Synthetic Data for Text Localisation in Natural Images Supplementary Material" IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

8. Zhi Tian, Weilin Huang, Tong He, Pan He, Yu Qiao, Detecting Text in Natural Image with Connectionist Text Proposal Network, University of Oxford, Sept 12, 2016.

9. Ronan Collobert, Laurens van der Maaten, and Armand Joulin Torchnet: An Open- Source Platform for (Deep) Learning Research, Facebook AI Research, 33rd International Conference on Machine Learning, New York, NY, USA, 2016.

10. Bouwmans T. Traditional and recent approaches in back- ground modeling for foreground detection: an overview. Comp Sci Rev 2014; 11: 3166.