Automatic Pothole Detection System

Automatic Pothole Detection System

S Gayathri Dept of ECE SJCE

Mysuru,India

Mamatha RG Dept of ECE SJCE

Mysuru,India

Manasa B Dept of ECE SJCE

Mysuru,India

Menita Patil Dept of ECE SJCE

Mysuru,India

Sanjana BM Dept of ECE SJCE

Mysuru,India

AbstractPothole is a depression in the normal surface of the road. Larger potholes sometimes cause breath holding accidents and loss of lives as they are not visible at night. Pathetic condition of roads is a boosting factor for traffic congestion and accidents. The driver must manually look for potholes on the road while driving sometimes, the driver encounters many risks such as he will be at a constant speed and suddenly there will be a pothole or hump on the way. At these times, the risks of accidents are more. This paper presents an prototype which detects the pothole in the path of driving and update it to the cloud. A voice notification alerts the driver of the potholes ahead. Website is built which will be accessible by the authorities concerned so that they can take care of the potholes detected. This system will help in the maintenance of the roads.

KeywordsIoT, Cloud, Clustering, Voice, haversine.

1. INTRODUCTION

   Roads are important means of transport which carry 90 percent of countrys passenger traffic. Major problem faced by developing countries is maintenance of roads. As we know most of the roads in India are narrow and congested with poor surface quality and maintenance of the roads are not satisfactory. Due to the poor maintenance and servicing of the roads has led to creation of potholes. According to a survey by automation association one of the major reasons for road accidents are potholes. When a driver slows down the speed of the vehicle there are high chances of collision with the vehicle following it. Hence we think information sharing plays an important role in avoiding the effects of potholes and reducing accidents.

2. RELATED WORK

   The existing methods for pothole detection can be divided into 3D reconstruction based, Sensor based, Vision based methods. In this section we present few methods that are relatively close to our work in this paper.

   Most of the systems used ultrasonic sensor to measure the height and depth of the potholes. In [1] microcontroller processes this information and triggers GPS module to send the location to the cloud database. Other vehicles are

   alerted via android app with an audio beep. Magnitude of reflected signal from the ultrasonic sensor decreases due to roughness of the surface and signal amplitude is difficult to analyse. So accelerometer is used to optimise these variations present in the signals [6]. It is difficult to classify large and small potholes using the weak signal from this sensor. The information about the pothole presence is transmitted to other vehicles in the vicinity via ZigBee module [2]. The transmitting vehicle speed is directly controlled by the microcontroller acting on the motor drivers. In [13] vehicle mounted accelerometer sensor is used. The sensor data and the location from the GPS are pre-processed and then passed to machine learning based damage detection method.

   In [4] 2D LiDAR (Light detection and ranging) and Camera are used. LiDAR obtains the angle and distance informations. Image based detection improve the accuracy, the algorithm includes noise filtering, brightness control, binarization, addictive noise filtering, edge extraction, object extraction and pothole detection. Image based detection methods have the advantage of not being affected by electromagnetic waves and road surface state. 0Video processing based road crack recognition method was simulated in MATLAB [10]. Here the video images are processed by grey transformation and image smoothening techniques. Mathematical morphology methods are used to classify cracks into categories like transverse, longitudinal and turtle crack.

   In other vision-based methods an optical device is mounted at a height of rear-view mirror of a survey vehicle to obtain the video data [14]. Histogram and closing operation is used to separate image from background and further noise is removed by median filtering. This proposed method has a better performance for bright images, rather than dark images. Another approach based on pothole texture extraction and comparison [18]. The system is trained based on the existing number of pothole texture samples such that it detects the potholes based on this training result. In [19] a method based on wavelet energy field has been implemented as a MATLAB prototype, that achieves a high detection and segmentation accuracy. Here, the wavelet energy field of the pavement image is

   constructed to detect the pothole by morphological processing and geometric criterions. And, the detected pothole is segmented by Markov random field model and the pothole edge is extracted accurately. In [15] the proposed system makes use of both vision and sensor- based methods, where an accelerometer identifies the pothole and mobile application is developed to capture the pothole image and sent to the cloud.

   3D reconstruction method makes use of accelerometer sensor present in the smart phone to capture the vehicle vibration. In the signal, z-axis corresponds to vehicle vertical vibration and x-axis corresponds to horizontal vibration. Gaussian model-based mining algorithm is use.0d is for abnormal event detection [7], x-z ratio filtering is applied for classification as pothole or hump. In

   [3] access points will be placed on roadsides for broadcasting data. All the vehicles are Wi-Fi enabled and whenever they are in the vicinity of any access point the information regarding the pothole will be sent. Here many access points have to be present along the stretch of the road which proves to be expensive. In [11] a simulation- based infrastructure for data transfer to the moving vehicles is proposed. It has access points and Wi-Fi equipped mobile nodes. Mobile nodes get data about road condition in their vicinity. Information is passed as packets, which is broadcasted over UDP.

   In [16] IR transmitter-based pothole detection method is proposed. Two IR transmitters operating at 38 KHz frequency are used in order to detect two types of potholes. One with less depth and is not dangerous. The other pothole is the dangerous one and which requires immediate repairing as the depth of pothole is large. When any one of the IR gets triggered, the microcontroller will be notified that the IR has sensed a pothole. The location data will be stored inside the memory of the microcontroller. The microcontroller then uses the GSM kit to send an SMS to predetermined number i.e., government body responsible for road maintenance. The SMS body consists of the exact location of that detected pothole. In [12] potholes, braking and bumps are detected using 3-axis accelerometer sensor. It makes use of 7 core modules the sensing module, threshold computing module, the artificial learning module, road condition detection module, traffic information module and the data aggregation module.

   Although the vision-based methods are cost effective compared with 3D laser scanner methods it may be difficult to accurately detect a pothole using these methods because of the distorted signals generated by noise in collecting image and video data. In proposed system [9] Bluetooth technology was used and in [2] Zigbee technology both of which have limited range of communications, slow data speed and short-term battery life. Zigbee technology requires high maintenance cost. Most of the proposed systems made use of ultrasonic snsor for detection. But during detection using ultrasonic sensor, the pathway which is affected by pothole is greatly influenced by scattering signal of ultrasonic sensor. So the magnitude of reflected signal decreases due to the roughness of the surface and the signal amplitude is difficult to analyse. All

   the above methods confirms the presence of pothole by collecting the data from a single vehicle our proposed system considers 5 vehicles which increases the probability of pothole confirmation.

3. SYSTEM REQUIREMENTS
   1. Hardware description

      The primary sensor used in the prototype is piezo- electric sensor. These sensors work on the principle of piezo electric effect i.e. electric voltage is produced on application of mechanical pressure. NodeMCU combines features of WIFI access point and station plus microcontroller and uses simple LUA based programming language. The ESP8266 is a low-cost Wi-Fi microchip with full Transmission Control Protocol/Internet Protocol (TCP/IP) stack and microcontroller capability. The SKG13BL is a complete GPS module that features with super sensitivity, ultra-low power and small form factor. The GPS signal is applied to the antenna input of module, the serial interface output NMEA protocol data or customer protocol data with position, velocity and time information.

   2. Software description

   Android Studio is the official integrated development environment (IDE) Google’s Android operating system, built on JetBrains’ IntelliJ IDEA software and designed specifically for Android development. The Arduino integrated development environment is a cross-platform application for Windows, cOS, Linux. It is used to write and upload programs to Arduino compatible boards, but also, with the help of 3rd party cores, other vendor development boards. SQLyog is the most powerful anager, admin and GUI tool for MySQL, combining the features of MySQL Query Browser, Administrator, MyAdmin and other MySQL Front Ends and MySQL GUI tools in a single intuitive interface. Microsoft Visual Studio is an integrated development environment (IDE) from Microsoft. It is used to develop computer programs, as well as websites, web apps, web services and mobile apps. Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud.

4. METHODOLOGY
   1. Hardware with website

      The hardware consists of piezoelectric sensor, GPS module and node MCU. The sensor gives the reading between 0 to 1023. Whenever the pressure or force is exerted on it, the exerted force is converted into electrical voltage. The sensor has two terminals, one of which is connected to ground and the other one is output terminal which is connected to Analog input pin of Node MCU. GPS module is powered with 3.3V from the Node MCU. The transmitter pin of GPS is connected to receiver pin of Node MCU. Hardware module makes use MQTT protocol to send data to the cloud. The sent data includes latitude, longitude and the sensor values. These data are published every 3 seconds and it can be received on any device which subscribes to that particular topic.

      Sqlyog is a GUI tool for the database in the cloud. To view the data we need to login using server IP and database name. Then we have created 3 tables in our database. These tables gives us data in readable and ordered format. The 3 tables include readings, alert and Recovered Pothole. Readings table has all the raw data from the sensor. This table consists of ID, latitude, longitude and sensor value fields. The ID is auto generated and other data is obtained from cloud. The raw data is processed by executing select query on the readings to group it based on the locations. This gives pothole count from each location. If the count is greater than 5, it confirms the pothole presence in that particular location. Alert table is obtained by applying clustering algorithm to the above processed data. Alert table consists of different clusters of range 100m. Each cluster has starting and ending location and pothole count within the cluster. This is used to alert the driver whenever he is in the vicinity of the pothole. Recovered pothole table consists of the clusters from which the potholes are repaired. The contents of this table will be no more present in the alert table.

      The application is developed in visual studio. The concerned authority has the control over this service. Once they start the service the readings are sent continuously to the cloud. As mentioned above the hardware sends the data in the range of 0 1023 and the mobile accelerometer sends decimal data in the range 0 to 1. Hence the hardware data is mapped into decimal values less than 1. In case any reading greater than 1 is obtained in the database, it is deleted automatically. Even when the service is not started, if the hardware or mobile app sends the data, it is not lost instead it is stored in the MQTT queue. Later when the service is started those data can be obtained. The desktop app is designed such that it consists two buttons Start/Stop service and Exit. It also has 3 text boxes which is used to display live data latitude, longitude and sensor value.

   2. Android App

   An android app is developed to notify the driver about the ahead coming pothole. On the android app there is a start button by pressing which the application will get started. Latitude and Longitude values are got through android Network location provider which determines user location using cell tower and wifi signals providing location information in a way that works indoor as well as outdoor. Vibration values are got from the accelerometer sensor of the mobile. The collected information is sent to the cloud through HTTP protocol. The values are stored in hash table. In the cloud data is analysed using Haversion distance algorithm to get the distance between the two latitude and longitude. The stored data is retrieved from cloud and sent to the app as HTTP response. If the distance is less than the threshold value set(<100m) then through text to speech API voice notification of pothole ahead is displayed on the app The service can be stopped by pressing the stop button.

   The fig.1, shows overall working of the proposed system. To the cloud simultaneously values from the hardware as well as from the mobile are sent. Clustering

   occurs in the cloud, vibration values below the threshold in the same location for more than five times are considered as potholes which can be seen through the web as well as android application.

   Fig.1. Block diagram of the system

5. RESULTS ANALYSIS

   The final model with all the mentioned features was implemented using necessary hardware and softwares. The module is placed inside the vehicle near the shock absorber to get precise and better values as the impact is more near the shock absorber. To check the pothole in the website one has to login to the page. To use the mobile application, mobile should be connected to internet and location should be enabled for the application in the settings.

   1. Desktop application

      Fig.2. Desktop app showing the Latitude, Longitude and sensor

      readings

      When the service is started, the readings from hardware and mobile app are continuously sent to the cloud as shown in figure 2. The desktop application shown in fig.2. which displays latitude, longitude and vibration values.

   2. Database

      Fig.3. Pothole count table

      The data in fig.3. shows the pothole count in each location. In certain locations the pothole is detected multiple times and in other locations it is detected only once. We consider the pothole presence only when the count is greater than 5. That is from certain location if more than 5 vehicles sense the pothole then it is confirmed that there is a pothole. Or else it can be a false data or just a crack on the road. Clustering of pothole can be seen in in fig.4. Each cluster has a starting and ending location and pothole ount within the cluster.

      Fig.5. Login page of web application

      Fig.6. Clusters formation

      Fig.6. gives the pothole count within each cluster after applying the algorithm to the data. If the concerned authority takes any action then those potholes are pushed into the recovered pothole child page as shown in the below fig.7.

   3. Webpage

      Fig.4. pothole table with cluster count

      Initially the concerned authority has to login into the web application. The web application can be logged in by entering specific Email and password which is seen in fig.5. The master page consists of two child pages namely Pothole count and Recovered Potholes. The concerned authority will be viewing the cluster location and its Pothole count. The action field contains two options, take action and set recovered. When authority people want to take necessary on pothole in a cluster, they click on take action button and as potholes are repaired set recovered is clicked as seen in fig. 6. This updates the database by removing the repaired cluster from the database and thus no alert will be given to the user.

      Fig.7. Recovered potholes

   4. Android application

   The user must primarily download the pothole app and enable the location in the settings of the app. To start the service user needs to press the start button. A text saying service started appears on the app. Once there are vibrations between 0-1 these values are displayed as fallThreshold values.

   Fig.8. Android Application displaying Latitude, Longitude and

   fallthreshold values

   Once there are vibrations between 0-1 these values are displayed as fallthreshold values seen in fig.8. At that instant of time app also displays the latitude and longitude of the location of the pothole. After clustering the values in the cloud if pothole has occurred more than 5 times in the same location then a voice notification appears saying There is pothole nearby go slow. The service can be stopped by pressing the stop button.

6. CONCLUSON AND FUTURE SCOPE

The implementation of the prototype was done after comprehensive study on the existing systems. The drawbacks of other existing systems which used ultrasonic, IR sensors and image processing is overcome by this system. It has been successfully tested practically by placing the prototype in vehicles. Clustering of potholes has helped in effectively identifying the potholes. The Website developed helps the concerned authority to rectify the potholes and voice notification makes it easy to alert the driver about the nearing pothole. The potholes which are removed from the roads get updated in the database. Thus it is said to be an interactive system. Hence, It can be concluded that idea of this system will help in maintenance of roads and providing safety to the people.

The main goal of the project of pothole detection has been practically implemented. The prototype that is built is primitive there is always scope for future improvements which can be in the hardware as well as software part. The following ideas can be considered for the future expansion of the model:

• Maps can be included in the android application for graphically showing the pothole location.
• Along with the existing voice notification There is pothole nearby go slow, Distance of the pothole can also be voice notified.
• A more complex prototype can be developed so as to control the speed of the vehicle when there is pothole ahead.
• More accurate and complex algorithm can be used to find the distance between two latitude and longitude.

REFERENCES

1. Rajeshwari Madli, Santosh Hebbar, PraveenrajPattar, and VaraprasadGolla “Automatic Detection and Notification of Potholes and Humps on Roads to Aid Drivers , IEEE sensors journal, vol. 15, no. 8, page no. 4313 – 4318, august 2015.
2. Shambhu Hegde, Harish V. Mekali, GollaVaraprasad ,”Pothole Detection and Inter Vehicular Communication “, IEEE International Conference on Vehicular Electronics and Safety (ICVES) December 16-17, 2014. Hyderabad, 2014,
3. Sudarshan S Rode, Shonil Vijay, Prakhar Goyal, Purushottam Kulkarni, Kavi Arya , “Pothole Detection and Warning System: Infrastructure Support and System Design “, International Conference on Electronic Computer Technology ,2009.
4. Byeong-hoKang , Su-il Choi, Pothole Detection System using 2D LiDAR and Camera IEEE sensors journal, VOL. 17, NO. 16, page no. 5192-5197.
5. X. Yu and E. Salari ,”Pavement Pothole Detection and Severity Measurement Using Laser Imaging “, Department of Electrical Engineering and Computer Science The University of Toledo .
6. Chellaswamy C, Anusuya T, Famitha H, Amirthavarshini S B, IoT Based Humps and Pothole Detection on Roads and Information Sharing , international conference on computation of power, energy, information and communication (ICCPEIC), page no. 84 – 90.
7. Harikrishnan P.M, Varun P. Gopi, Vehicle Vibration Signal Processing for road surface Monitoring , IEEE sensors Journal, Vol. 17, No. 16, 5192- 5197, august 15,2017.
8. Jose M, Alvarez, Antonio M. Lopez, Theo Gevers, Felipe Lumbreras

   , Combining priors, appearance and context for road detection IEEE transactions on intelligent transportation systems, vol. 15, no. 3, june 2014.

9. Samir Patel, Amit Kumar, Pradeep Yadav, Jay Desai, DipaliPatil

   ,Smartphone-based Obstacle Detection for Visually Impaired People,International Conference on Innovations in Information, Embedded and Communication Systems (ICIIECS),2017.

10. Gang Shen, Road Crack Detection Based on Video Image Processing, The 3rd International Conference on Systems and Informatics (ICSAI 2016), pg no. 192 – 917,2016.
11. Sudarshan S. Rode, Shonil Vijay, Prakhar Goyal, Kavi Arya, Pothole Detection and Warning System: Infrastructure Support and System Design, 2009 International Confeerence on Electronic Computer Technology, page no. 286- 290.
12. Victor Akinwande, Olayiwola W. bello, Adewole K.S.

    ,AbimbolaAkintola, Automatic and real-time Pothole detection and Traffic monitoring system using Smartphone technology, Conference Paper , august 2015.

13. Ali Anaiss, Nguyen Lu Dang Khoa, Thierry Rakotoarivelo, MehrisadatMakkiAlamdari, Yang Wang, Smart pothole detection system using vehicle-mounted sensors and machine learning, Journal of Civil Structural Health Monitorinng, February 2019, volume 9,Issue 1. pp91-102.
14. Seung-Ki Ryu, TaehyeongKimand and Young-Ro Kim, “image based pothole detection system for ITS service and road management system”, Hindawi Publishing Corporation Mathematical Problems in Engineering Volume 2015, Article ID 968361.
15. Rm, Kabilesh, Automatic detection of road pothole using Raspberry pi and mobile application. 2018, Article 10.13140/RG.2.2.33184.89604.
16. Kathane, Samyak&Kambli, Vaibhav& Patel, Tanil& Kapadia, Rohan. (2015). Real Time Pothole Detection and Vehicle Accident Detection and Reporting System And Anti-theft (Wireless), International Journal of Engineering Trends and Technology. 21. 187-190. 10.14445/22315381/IJETT-V21P232.
17. Bychkovsky, hull, V Zhang, Y. Chen, K Goraczko, M Miu and Madden S. (2010 october) CarTel a distributed mobile sensor computing system, In proceeding of the International conference on Embedded networked sensor system, pp125 18 ACM.
18. C Koch, I. Brilakis, Pothole detection in asphalt pavement image,Advanced Engineering Informatics, Vol. 25(3), pp. 507-515, 2011
19. Wang, Penghui& Hu, Yongbiao& Dai, Yong & Tian, Mingrui. (2017). Asphalt Pavement Pothole Detection and Segmentation Based on Wavelet Energy Field.Mathematical Problems in Engineering. 2017. 1-13. 10.1155/2017/1604130.