Detection of Potholes using Machine Learning and Image Processing

Detection of Potholes using Machine Learning and Image Processing

Prof. Vijayalakshmi B

Asst. professor, Department of Computer Science Vivekananda Institute of Technology

Bengaluru, India

Kiran P

Department of Computer Science Vivekananda Institute of Technology Bengaluru, India

Madhusudhan GR

Department of Computer Science Vivekananda Institute of Technology Bengaluru, India

Kishor Jadav B

Department of Computer Science Vivekananda Institute ofTechnology Bengaluru, India

Manoj KS

Department of Computer Science Vivekananda Institute of Technology Bengaluru, India

Abstract- Streets are intended for explicit paces at which vehicles can go with security. Not with standing, at specific areas, for example, kept an eye on and automated level intersections, sharp curves, congested or accident- prone, extra control of speed might be important to guarantee wellbeing. This accomplished through a few different ways, for example, posting obligatory speed limit signs, utilization of blazing signals to alarm drivers, and street markings. In specific circumstances, street bumps are required, however their utilization on streets isn't viewed as acceptable designing practice. Plain speed breakers so not fill the need, however harm the vehicle and in not many cases they have caused serious wounds and passing. Traffic signs may likewise divert the driver and take their consideration from the street. Anyway absence of light or over the top light makes it hard to perceive the speed breaker ahead. This report talks about the technique for distinguishing mounds and potholes and gives ideal cautions to drivers to stay away from mishaps or vehicle harms. A picture preparing framework is utilized to distinguish the mound. We have additionally utilized ultrasonic sensors which are utilized to recognize the potholes by estimating the profundity of the streets individually. The proposed framework likewise stores the geographic area of the recognized pothole to the cloud utilizing IO T. This data fills in as important source to the administration specialists.

KeywordsHump Detection, PotHole Detection, Animal Detection, Convenutional Neural Networks Algorithm, Haar Cascade Algorithm.

1. INTRODUCTION

   India, the second most famous Country in the World and a quickly developing economy. HRoads are the prevailing methods for transportation in India. Be that as it may, the greater part of the streets in India are narrow and congested with helpless surface quality and street upkeep needs are not fulfillment met. Regardless of where you are in India, driving is a breath-holding, multi-reflect including, conceivably life undertaking. Throughout the most recent two decades, there has been a gigantic increment in the vehicle populace. This expansion in the quantity of street.

   In the course of the most recent two decades, India has developed hugely, as an ever increasing number of individuals graduate continuously and increasingly more of us gain work constantly ,we are will undoubtedly drive and invest the greater part of our energy voyaging. Equal , reports and study are of the feeling that [1]. "Last year,10,780 individuals were murdered in crashes brought about by potholes ,speed breakers and streets under fix or being developed. Despite the fact that fatalities under these catagories had descended possibly from 2014,the number of individuals murdered because of potholes rose to 3,416 from 3,049 in the past years."Evidently the key motivation to street mishaps happens to be flawed streets and unexpected occurance of potholes. The wellbeing of the drivers ought to be organized and a smooth drive ought to be guaranteed for everybody. Thinking along such lines we thought of coordinating an equipment framework which distinguishes the potholes utilizing an accelerometer and a spinner alongside a product which can quantify these outcomes on an ongoing premise and store the essential information on a cloud base.

   In this quick moving world that we live in, safe drive isn't just everybody's need yet additionally to givean issue free transport between places is the administration's obligation. In this paper, Here we propose a framework which recognizes potholes out and about. As we probably am aware avoidance is superior to fix, so we structure and execute a framework which perceives potholes as well as stores this information on a cloud stage which go about as a database for additional reference and empower us to dissect the information. The proposed framework contains two significant capacities, first to identify the pothole which is done through a multi-sensor subsystem comprising of accelerometer and whirligig and afterward also warrn the driver store this data on a cloud base which can be gotten to by different clients which will assist them with securing the potholes on their way. When the area of the potholes is known for us, The Government specialists can be educated about the equivalent.

2. PROBLEM STATEMENT

   Potholes have been a difficult issue and have become a danger forsafe street travel, so as to beat the issue we are proposing this framework utilizing MI(Machine Learning) and Image Processing. which will recognize the potholes and by doing so we can proficiently handle the issue.

3. SCOPE AND OBJECTIVES

   Pot holes are a heavy issue andbecame a

   threat for safe road travel, so as to beat the problem we are proposing this technique using machine learning and image processing, which can detect the pot holes and by doing so we will efficiently tackle thedifficulty.

   • Better road safety: Machines aren't at risk of human-error and distractions, resulting in swift and appropriate responses in real- time road conditions.

   • Reduced commute time: With vehicles communicating through one another and

     These directions give you essential you rules for getting ready camera-prepared papers for McGraw- Hill gathering procedures/Journal Publications.

     In India ordinarily streets have speed breakers so the vehicle's speed can be controlled to keep away from mishaps. Be that as it may, speed breakers are unevenly appropriated with lopsided and informal statures.

     Potholes, shaped are because of substantial downpours and development of the overwhelming vehicles, additionally become a significant explanation behind mishaps and loss of human lives. As indicated by the review report of us "Street Accidents in India, 2011", an aggregate of 1,42,455 individuals had lost their carries on with because of street mishaps. Of these, almost 1.4 percent or about 2,200 fatalities were because of helpless state of streets. To address the previously mentioned issues are, a financially savvy arrangement is required that gathers the data about the potholes and bumps and furthermore causes drivers to drive securely. With the proposed framework has been made to underwrite drivers to avert the mishaps caused because of potholes and raised protuberances.

     using modern GPS systems, commute times is greatly reduced as self-driving vehicle decreases the "phantom effect" in modern-day traffic.

   • Increased productivity: Reduced commute times mean longer are often spent on what depends more.

   • Reduced expenditure: Reduction in accidents will directly cause reduced expenditure on damages.

   • Environment-friendly: Efficient performance driving forms of the self- driving car will result in loweremissions.

4. IMPORATNCE OF THE PROJECT Improving the safety of traffic is an Important issue

   Indin telecomservices (ITC) and the potholes on the road

   causes serious harmto drivers safety. This technique can be used detect potholes with lower cost in a complete environment. This study proposes a potholes detection method based on the machine learning and imageprocessing.

5. SOFTWARE ANDHARDWARE

   REQUIREMENTS

   • SD-card It is mainly used to providehigh capacity memory in a small size.

   • LCD Display It is used to produce a visible image. It allows the display to be much thinner when compared to cathode ray tube technology.

   • Raspberry Pi : It is the device that enables people of all ages to explore computing, and to learn how to program in languages like scratch and python.

   • Camera It is the optical instrument used to record images.

   • Ultrasonic sensor It uses high frequency sound waves to resonate a desired frequency and convert electric energyinto acoustic energy, and vice versa.

   1. HARDWARECOMPONENTS

      1. RASPBERRYPI

         The Raspberry Pi could be a small low cost single board computer having a processor speed starting from 700 MHz to 1.2 GHz for the Pi 3.The on-board memory ranges from 256 MB to 1 GB RAM. The boards supports up to 4 USB ports together with HDMI port. Along from all this it's number of GPIO pins which support protocols like I²C.Moreover it also supports Wi-Fi and Bluetooth facility which makes device very compatible with other devices. It supports Scratch and Python programming languages. It supports many operating systems like Ubuntu MATE, Snappy Ubuntu, Pidora, Linutop and plenty of more out of which Raspbian is specifically designed to support Raspberry Pis hardware.

         Fig 1. Raspberry Pi

      2. PI CAMERA

         Pi camera is great gadget to capture time- lapse, pic with great video clarity. It connects to Raspberry Pi via a versatile elastic cord which supports serial interface. The camera image sensor incorporates a resolution of 5 megapixels and features a focused lens. The camera provides a good support for security purpose. Various characteristics of the camera are it supports 5MP sensor, Wide image, capable of 2592×1944 stills, 1080p30 video on Camera module v1.

         Fig 2. Pi Camera

      3. ARDUINO MICROCONTROLLER

         This microcontroller relies on ATmega329P.There are 14 digital input/output pins available out of which 6 are often used a PWM outputs. It also supports 6 analog inputs. it's 16 MHz quartz, a USB connection, an influence jack, an ICSP header and a push button. it's 32 kb of non-volatile storage and a pair of kb of SRAM and weighs around 25g [13].Apart from of these features Arduino IDE is extremely user friendly and uses basic as its artificial language

         Fig 3. Arduino Controller

      4. ARDUINO IDE

         Arduino IDE is that the platform were the programs are written for Arduino board. it's compiled button which helps in compiling the code together with the upload tab which helps to upload the code on the board. Programs written on Arduino IDE are often called Sketches and are saved as ion extension. The editor has numerous other features like verify, save, upload include library and serial monitor. except this, the developers have made easy to use functions, which makes coding easy and fun. Moreover, there are number of examples provided for every and each interface which helps the user learn more about functions and hardware yet.

      5. OPENCV AND ANACONDA ENVIRONMENT

         OpenCV is an open source computer vision library which is capable of handling images/videos from fairly basic tasks to utter complex tasks like automatic face recognition. It supports C++, C, Python and Java programming languages and supports Windows, Linux, Mac OS, iOS and Android. Written in optimized C/C++, the library can advantage of multi-core processing. Enabled with OpenCV, is the benefit of the hardware acceleration of the underlying heterogeneous calculate platform. during this project it's serving a big support, it helps to basset the section of the video from the Raspberry-Pi cam interface as shown above and converts it to the grayscale, resize it then passes it to the synthetic Neural Network. Anaconda may well be a strong interactive development environment for the Python language which has advanced editing, interactive testing, debugging and introspection features and a numerical computing environment. it is a matplotlib as plotting library which helps to plot 2D/3D graphs

      6. RASPBERRY PI CAM INTERFACE

      To remotely capture the live feed from the camera to the laptop we need to develop an interface which would serve this purpose. This is where the software pi- caminterface comes into the picture. Its the program which helps you capture the live feed by just letting the IP address of the Raspberrypi. One can record and download video/image in various resolutions with different number of settings. Below is the view of the software under action. The car was trainedunder different combinations of the track i.e. straight, curved, combination ofcurved and straightand etc. Total of number

      24 videos were recorded out of were the images extracted.10868 images were extracted and was categorically placed in different folders like left, right, straight and stop. Below is the sample image of each of the scenario in its gray scaled version.

6. SYSTEM DESIGN

   In system design we need to have mainly about the architecture, modules, components and the data we need to be added for a suitable architecture.In system design the system theory would be seen as the application for the development of a product. In computer system design the most widely used methods are object oriented analysis and its methods are being used.To satisfy the specified requirements of the user , the process of defining and developing the system is the process of systemdesign.In the object oriented analysis and design system architecture UML will be a standard language which is a conceptual model that is going to define the behavior and structure of the system.It is going to combine the relationships andsystemcomponents that they are going to describe how its going to work to implement the over allsystem.

   1. ARCHITECTURE

      Fig 4. System Architecture

   2. DATA FLOWDIAGRAMS

      The graphical representation of the flow of data through the information processing system modelling its the process aspects is the data flow diagram. The data flow diagram can be used as a preliminary or a first step to create the overview of the system without going into the depth of the detail and later in such a way it can be gone depth into the process. The visualization of data processing can be one of the advantages of the data flow diagram. The main aim of data flow diagram is that it shows what the type of information should be taken as the input and the output will be taken from the system, and howthe data will be assessed and stored will be seen in the data flow diagram.

   3. SEQUENCE DIAGRAM

      The object interactions which are arranged in time sequence shows the sequence diagram. There will be a scenario in which it involves and depicts the objects and classes and the messages is going to exchange between the objects which is going to carry out all the functionalities based on a suitable scenario. They are going to get associated typically some cases are being used with the development of the system under the view of logical process..

      The event diagram or the event scenario is also called as the sequence diagram. The sequence diagramconsists of the objects, different processes and the vertical parallel lines that they are going to occur one after the other simultaneously and based on the occurence the messages are going to exchange between them based on the horizontal arrows they are going to occur. In the graphical manner the simple runtime scenarios allows the specification process.gt5v4

   4. SEQUENCE DIAGRAM

      The object interactons which are arranged in time sequence shows the sequence diagram. There will be a scenario in which it involves and depicts the objects and classes and the messages is going to exchange between the objects which is going to carry out all the functionalities based on a suitable scenario. They are going to get associated typically some cases are being used with the development of the system under the view of logical process..

      The event diagram or the event scenario is also called as the sequence diagram. The sequence diagramconsists of the objects, different processes and the verticalparallellines that they are going to occur one after the other simultaneously and based on the occurence the messages are going to exchange between them based on the horizontal arrows they are going to occur. In the graphical manner the simple runtime scenarios allows the specification process.gt5v4

   5. SENSOR BASED CONTROL

      Fig 5. Sequence Diagram

   6. IMAGE BASED CONTROL ( ANIMAL

      /PERSON)

      Fig 6. Sequence Diagram of Animal Detection

   7. USE CASE DIAGRAM

      In the use case diagram its going to identify the other types of the diagrams which are going to be accompanies by the other types of the diagrams. The use case diagramis going to give the complete details ofthe work. It is going to provide a higher view of the system as it is said based on the use case diagramwe are going to see the real work and it provides the graphical representation and in the simplified manner it shows that how it actually works

      Fig 7. Use Case Diagram

7. IMPLEMENTATION

The implementation of a system is combining the individual modules and making the systemto work as whole in order to assure that all models works together interactively.

1. ANIMAL DETECTION

   From the deep neuralnetworks we are going to obtain the correct predictions and to get it correctly the data should be preprocessed.

   1. Mean subtraction

   2. Scaling by some factor

      We are going to use the deep neural network which contains the following two basic functions such as the preparingandpreprocessing of images.

      1. BLOCK DIAGRAM OF ALGORITHM

         Fig 8. Block diagram of algorithm ALGORITHM:

         Step 1: Image/video acquisition from the camera. Step 2: Convert video to frames.

         Step 3: Store images of each animal as database which is used as training set for our program.

         Step 4: The database is going to compare with the captured images.

         Step 5:Use im read function to read the image and Preprocessing is done on that image. Perform Blob detection on the frame and blobs are matched with images from training database images.

         Step 6: And check if it is matching or not.

         Step 7: To identification of that animal is desired or not. An array is created and program is written for each animal to be identified.

         Step 8: Intimation or alert

2. PRE-PROCESSING

   The input image type and feature extraction depends on the image processing.

   Some common methods are:

   • Denoising: applying a Gaussian or simple box filter for de noising.

   • To increase the speed we use down sampling.

   • Binary images uses morphological operations.

   • Some factor uses scaling method..

3. IMAGE SEGMENTATION

   The image segmentation used in this is a threshold segmentation. To put it simply, the threshold of the grey scale image segmentation is to identify a range in the image of the compared with the threshold and accordingly to the results to the corresponding pixel is divided into two categories, the for ground and background.

   • The threshold will be determined.

   • The threshold value will be compared with the pixel value.

4. POT HOLE DETECTION

   Fig 9. high-level design

5. ALGORITHM ( METHODOLOGY)

   In this WORK, a visualapproach is proposed that does not require any machine learning algorithms in the same fashion as the related work presented in the previous section.

   Fig 10. methodology

   In our work we are going to fix the camera inside the vehicle around some angle of 120 degree. Based on the camera resolutions we are going to get the clarity of the images as well as we have used the 5mp camera for our work and when the resolution increases we are going to get a blur image and if the resolution decreases we are going to get a very good picture without any kind ofblur.

   1. RESULTS

6. POTHOLE DETECTORMODEL

A model which is used to detect a potholes and it consists of the following parts such as raspberry pi, arduino and power supply which is used to detect a potholes.

Fig 11. Model which detects the Potholes

1. POTHOLEIMAGE

   This is the one among the image or the data what we have been collected and it is the pothole image where the image is first converted togrey scale imageusing suitable algorithms and givesthe correct result.

   Fig 13. Output of Pothole Detection

2. HUMAN DETECTION

   In this detection the humans are going to detect and the accuracy will be based on the what type of camera and based on the lens or megapixel its going to give the accuracy of the image.

   Fig 15. Detection of Human

3. ANIMALDETECTION

   By using the suitable algorithms the animal image are going to be detected. And these are the techniques in which thedetectionofanimals has been done.

   Fig 16. The dog image which needs to be detected

4. ANIMAL DETECTIONOUTPUT

Fig 17. Output of animal detection

1. CONCLUSION

   By using the neural network configurations and the software the hardware components are going to describe properly. Our model was based on the machine learning and image processing model was completely developed. The social barriers should be overcome by the autonomous vehicle and based on the advancement of the technology the metal models will be influences based on the automobiles. Now a days for these cars a new legislation is going to create a new opportunities. Our work is going to be based on the driverless cars

2. FUTURE ENHANCEMENTS

   In a single model there will be a development of the traffic signal detection, lane detection and all the detections will be done in a very single model. For security purposes they are going to create a protective barriers, it is going to increase the potential vulnerabilities, the usage of the bandwidth is going to be very efficient. They are going to provide a specialization and abstraction. There is going to be a one layer that misses the security, deployment and prototyping link between all the devices.

3. REFERENCES

[1]. Sukhad Anand ,Gunjan Chugh, Divya Bansal and Sanjeev Sofat, Road Condition Detection Using Smartphone Sensor: A Survey, International Journal of Electronic and Electrical Engineering.

[2]. Jin Lin and Yayu Liu Ajit Danti, Jyoti Y. Kulkarni, and P.

S. Hiremath, An Image Processing Approach to Detect Lanes, Potholes and recognize road Signs in Indian Roads, International Journal of Modeling and Optimization, Vol. 2, No. 6, December2012.

[3]. Kiran Kumar Vupparaboina and Roopak

R. Tamboli "Overtaking & receding vehicle detection for driver assistance and naturalistic driving studies." In Intelligent Transportation Systems (ITSC), 2014 IEEE 17th International Conference on, pp. 697-702. IEEE, 2014.

[4]. Zhajian Li I. Kolmanovsky, E. Atkins, J. Lu, D. Filev, and

J. Michelini, Cloud aided safety- based route planning, in 2014 IEEE International Conference on Systems, Man, and Cybernetics, pp. 25252530, October 2014.

[5]. Rui Fan H. Youquan, W. Jian, Q. Hanxing,

Z. Wei, and X. Jianfang, A research of pavement potholes detection based on three-dimensional projection transformation, in Proc. 4th Int.Congr. Imge SignalProcess (CISP), Oct. 2011, pp. 1805 1808

[6]. I. Moazzam, K. Kamal, S. Mathavan, S. Usman, and M. Rahman, Metrology and visualization of potholes using the microsoft Kinect sensor, in Proc. 16th Int. IEEE Conf. Intell.Transp. Syst., Oct. 2013, pp. 12841291

.[7]. Sachin Bharadwaj Sundra Murthy and Golla Varaprasad, Automatic Detection and Notification of Potholes and Humps on Roads to Aid Drivers, IEEE Sensors J., Vol. 15, No. 8, August 2015.

[8]. Zhaojian LiI. Kolmanovsky, E. Atkins, J. Lu, D. Filev, and

J. Michelini, Cloud aided safety- based route planning, in 2014 IEEE International Conference on Systems, Man, and Cybernetics, pp. 25252530, October 2014.

[9]. Vijaya Kumar Samyak Kathane, Vaibhav Kambli, Tanil Patel and Rohan Kapadia, Real Time Potholes Detection and Vehicle Accident Detection and Reporting System and Antitheft (Wireless), IJETT, Vol. 21, No. 4, March 2015.

[10]. Van Khang Nguyen and ´ Eric Renault, Taehyeong Kim, Seung-Ki Ryu, A Guideline for Pothole Classification, International Journal of Enginering and Technology (IJET), Vol. 4, No. 10, October 2014.

Volumes8in, cIsesutehe15introduction of the automobiles Paunbeliwshleyd by, www.ijert.org

developed cars are going to develop a potholes.

181