Detection and Quantification of Road Surface Damage using Digital Image Processing Techniques

Detection and Quantification of Road Surface Damage using Digital Image Processing Techniques

1. Mohan N, Asst. Prof, 2.Bhavani N, 8th sem, 3.Kasthuri K, 8th sem,

4. R A Varun,8th sem, 5. Bhavya M,8th sem,

Dept. of ECE, Dr. TTIT.

Kolar Gold Fields, Karnataka 563120

Abstract Road transportation is the main form of transportation in any given region. Whereas the road damages that occur due to the natural disasters or the ill maintenance of the roads, will eventually lead to the delay and also results in the horrible travel experience, which may not be suitable in certain situations. In this paper we propose a method to detect the damages on the road surface using the videos of roads in different regions. The damages on the road surface are quantified from the video obtained using a simple algorithm. As precautionary measure, a warning message will be displayed so that the traveler can choose an alternate route. The experimental results shows that the efficiency and accuracy of the proposed method.

Keywords Road surface damage, potholes, warning message, damage in terms of area.

1. INTRODUCTION

   Road transport is the most common and complex network as it covers a wide range, physically convenient, highly flexible and usually the most operationally suitable and readily available means of movement of goods and passenger traffic over short, medium and long distances. Road extraction from high-resolution remote sensing image is active in recent years. Remote sensing technology is a cost-effective manner to identify the situation of roads, bridges and other ground transportation, and then determine the road traffic conditions. Road has obvious geometric features. As a technique widely used in computer vision field, has outstanding performance in detecting geometrical features hence, aiming to implement the extraction of road network and detect the damaged roads.

   Remote sensing images can be utilized for carrying out a quick road damage assessment [2][5]. But, extracting a road map from a remote sensing image is not a simple task, since there is no generalized mathematical model for representing the roads uniquely, because of lots of variations in pixel color depending on the materials utilized, the age of the road, weather conditions, image resolution and height and attitude of the sensor, when the picture was taken. Moreover, there are wide variations in roads, such as rural, urban, suburban, etc. In urban areas, especially the road types are different, and the road network changes at a rapid rate due to urban development, and to maintain an accurate

   and precise road network database is very difficult, time consuming and expensive.

   Road damages are detected mostly with the pre- damaged images of the same location. But, the roads as vectors need to be compared before and after the road was damaged, which is a more time consuming process and requires more space for storage. Earlier a team had [1] analyzed road damage by extracting the various parameters of damaged road, such as the starting and ending coordinate points, and its length, and width. Unfortunately, for irregular shaped damaged roads, some predefined values such as the maximum width, length and direction of road need to be given. The values of these parameters will differ from image to image [4]. To overcome this drawback, the proposed algorithm using digital image processing technique computes the road characteristics automatically from a given image; it does not require any predefined parameters.

   In road pothole is a kind of structural damage. Pothole detection plays an important role in highway administration and the maintenance department. Traditionally, pothole detection mainly relies on manual work, which is labor-consuming, time consuming, imprecise and dangerous [9]. Some systems use automatic algorithms for pothole detection [3], however high success in terms of classification rate has not been achieved due to lighting conditions, various in road texture and other difficult environmental conditions.

   Therefore, it is necessary to propose a kind of fast and effective method to improve the efficiency of detection. There have been many thoughtful studies on pothole auto recognition. In image preprocessing stage, many researchers have paid great attention. For example, Histogram equalization and spatial filtering [2] have been used to achieve the image enhancement. For image segmentation, a suitable threshold is needed, in a solution is proposed which is to break the image into tiles.

   The block size is a very important parameter. It should be large enough to contain both the crack and background regions while representing only the local information. After binary image processing, as noise and texture existed, there will be some isolated dots, burrs and small connected domains. Mathematical morphology is introduced into the applications of road image analysis and processing. But the result of them is not very well due to

   different lighting conditions and surface texture. Usually, even mathematical morphology method could not solve the problem that there are many breakpoints in the potholes. However, the algorithm is complex and time-consuming. In this project a novel method is introduced which carried out in two stages. First stage, the preprocessing stage is image segmentation and noise removing. The second stage is to detect the breakpoints of the potholes.

   Computer vision and mobile computing are powerful tools with great potential to enable a range of assistive technologies to aid growing population and transportation.

2. PROPOSED METHODOLOGY

   A flow diagram of approach to the proposed methodology is shown below. The method consists of two stages, they are preliminarily the obstacle detection through the video processing while the second phase consists of Road surface analysis for damaged or defected road detection and calculation of the area affected or damaged [fig 1],where as the proposed method is completely the second step of processing the single frame of the video.

   Fig 1: flow diagram for damage detection

   Obstacle Detection

   Detecting and counting obstacles can be used to analyze road surface and the traffic pattern. It is also a first step prior to performing more sophisticated tasks such as tracking or categorization of obstacles or objects by their size. This method shows how to use the foreground detector

   [1] and blob analysis to detect and count obstacles or objects in a video sequence and highlight them with the green box. This method assumes that the camera is either stationary or in motion. This method focuses on detecting objects.

   Step 1 – Import Video and Initialize Foreground Detector

   Rather than immediately processing the entire video, this method starts by obtaining an initial video frame in which the moving objects are segmented from the background. This helps to gradually introduce the steps used to process the video.

   The foreground detector requires a certain number of video frames in order to initialize the Gaussian mixture

   model. This example uses the first 50 frames to initialize the first three Gaussian modes in the mixture model

   Step 2 Detect objects in an Initial Video Frame

   The foreground segmentation process is not perfect and often includes undesirable noise. This method uses morphological opening to remove the noise and to fill gaps in the detected objects. Next, we find bounding boxes of each connected component corresponding to a moving object or stationary object by using computer vision. Blob Analysis of an object is performed to resize ad highlight the box around the objects detected. This method further filters the detected foreground by rejecting blobs which contain fewer than minimum specified pixels. To highlight the detected obstacles, the program draws green boxes around them.

   The number of bounding boxes corresponds to the number of objects or obstacles found in the video frame. It displays the number of found objects or obstacles in the upper left corner of the processed video frame within a rectangular yellow polygon box.

   Step 3 – Process the Rest of Video Frames

   In the final step, to have the continuity of the video without any interruption and its free flow the loop is created to process the remaining video frames. The output video displays the bounding boxes around the objects or obstacles. It also displays the number of objects or obstacles in the upper left corner of the video.

   Road Surface Analysis and Damage Detection

   The above specified methodology first starts from the acquisition of the image. The image can be acquired by many means of platforms. The resolution of the image the height of the acquisition device from the road plays an important role in the image processing. Whereas, the resolution gives the clarity of the image, the height determines the length of the road covered. For shorter road lengths the images can be captured from low heights, for the longer lengths of the roads, the images of the roads should be taken from the high altitudes. Satellite images depend on length, height, altitude, area coverage, etc.

   In road image the exact clarity which is required for processing cannot be obtained. Hence the alternate solution is to capture the road video using a drone. So, that the acquired input from the drone video is converted into number of frames. These frames give the required resolution and accuracy to process the image further.

   Pre-Processing

   In image preprocessing, image data recorded by sensors on a satellite restrain errors related to geometry and brightness values of the pixels. These errors are corrected using appropriate mathematical models which are either definite or statistical models.

   Once the image is acquired, it is subjected to the preprocessing. This preprocessing involves various aspects such as the removal of the unwanted regions within the images, such as the surrounding vegetation, or the

   environment. This step is done to minimize the executing time of these regions during the processing, which may delay the processing and correspondingly contribute to the lengthened executing time. As acquired color image consist of 3 planes (Red, Green, and Blue); it is difficult to process it in quick time. So, it is first converted into Grayscale image. After color to grayscale conversion, pre-processing stage uses some enhancement techniques to eliminate challenges created by noise, blurring effect and uneven lighting. Where the pixel range is reduced, the hue and the saturation region are removed and the luminance of the image is retained by converting the image from the true color image to the grayscale.

   Image Enhancement

   Image Enhancement is the process of manipulating an image so that the result is more suitable than the original for specific applications. The main goal is to emphasize (highlight) certain features of interest in an image for further analysis [6].

   Contrast Enhancement

   Once the gray scale image is obtained, it is perfected for the luminance; it performs the adjustment of the luminance of the image plane. Intensity adjustment is a technique for mapping an image's intensity values to a new range. In intensity adjustment image the pixel values which have less brightness becomes brighter and the pixel values with less darkness becomes darker. Hence it increases the contrast of the gray scale image.

   Image Filtering

   An image is often corrupted by noise during its acquisition or transmission. Noise is any undesired information that contaminates an image. Noise appears in images from variety of sources. The goal of image filtering is to remove/eliminate the noise present in images without eliminating certain important image features. Denoising can be done through filtering. It is considered that the acquired images can be affected by Salt and Pepper noise so before processing this image, noise has to be removed first. Following section explains one of the techniques to remove the noise.

   To eliminate the noise present in the image, which is not necessary and has nothing to contribute to the road and analysis of its surface, the image is converted into binary form, where the image is more easily represented in the form of the zeroes and ones. Now the image is represented in the logical form, the threshold is adjusted to make the image does not lose its characteristic components of the road and its surface, during the gray to the binary conversion. This step converts the grayscale image in to a binary image. The output image replaces all pixels in the input image with luminance greater than level with the value

   1 (white) and replaces all other pixels with the value 0 (black).

   Thus converted binary image, now comprises of the image in form of the black and white pixels, where the noise or the surrounding pixels which are now been represented in the form of the white pixels. To reduce these noise elements in the image, we go for the morphological functions. Morphologically open binary image function is used to remove the small objects, present in the binary image. It

   removes all connected components (objects) that have fewer pixels which may be specified, and producing another binary image, without those removed objects in the reproduced image. There is also a few default connectivity for the image, which is taken in general when no pixels size is specified.

   Since the road will be represented in the white pixels and the damages to it will be represented in the black pixels. During this operation, the damaged road pixels will have no effect from the small objects removal, as the entire road is represented in the white connected pixels

   As the most of the noise that is represented in the white pixels are removed now, the image will be smoother in other regions and it is represented in the form of black pixels.

   Now all the noise pixels are replaced by the black pixels, which are not considered and are considered as the background.

   To find out the amount of the region of the road that is damaged we go for the measuring [7] of the properties of the image region function by calculating the sum of the connected components functions. The undamaged road that is calculated by taking the number of the pixels present in the frame now. As now all the major noise elements are cleared and hence the number of pixels gives out the undamaged road in the frame. To calculate the damaged road pixel that is in the black color embedded within the white color pixels.

   Now to calculate the damaged road part, the image is negated, through negation the black becomes white and vice versa. Now there are two major components in the processed image in large white color on either side of the road representing the background. This two components always being greater than the road components. Are removed by selecting them as the larger components.

   Once the largest component is removed from the frame the remaining second component on the other side of the road is also removed in the same way. Here we have to remember one thing that the biggest component can emerge on any side of the road depending of the position of the camera while recording the frame.

   Once we have removed both larger components the remaining white dots and the connected components are nothing but the minute changes in the road to the potholes that are causing discomfort to the transport system through it.

   Now the damaged road is calculated by adding up all the white pixels although some noise components may be present in the form of white pixels this cause no harm to the sum of the damaged road pixels as they are negligible compared to the sum of the damaged road.

   An d then through negation of the processed binary image the damaged area in the road is calculated now as they are represented in white pixels, and as the white pixels representing any noise left will not contribute to the road damage as they will be negligible. The average surface damage to the road will be obtained by finding these two details. Then obtained details are processed for the calculation of the percentage of the damage occurred or present in the road. If the amount of road damaged is more

   than that of the threshold amount of road damaged fixed to trigger the message to the user/driver and statistics are obtained. And then based on the damaged road statistics the user/driver will be advised to either continue the road or to choose an alternative path [8].

3. EXPERIMENTAL RESULTS

   The above algorithm is executed and the damage was detected in the image. The resultant images are also shown below. If the damaged road pixels percentage is greater than that of the fixed percentage or threshold then the warning box or the message box is generated to choose the alternative path for better traveling experience. During the experiment a small amount of noise which became non removal but had minimal effect on the result, and hence it was neglected. Various sources of images were tested and the accuracy remained around ninety percent. The simple code that could provide the results faster and better and accurate based on the real time or in the offline using the prerecorded videos.

   1. [b]

   Fig 2: images showing the uniform intensity adjusted image[a] and the binary converted image[b].

   1. [b]

      Fig 3: images showcasing elimination of noise from the binary converted image[a] and its negated form[b].

      1. [b]

   Fig 4: Images showcasing elimination of sides[a] and the damaged road segments[b].

   The experimental results that was conducted on an aerial image of an remote area, is as shown in the table[Table 1].

   Parameter	Value[pixels]
   Undamaged road pixel	124383
   Damaged road pixel	12057
   Total road pixel	136440
   Result	Continue road

   Table 1: experimental results

4. CONCLUSION

   A video showing the road is taken and it is processed frame by frame. While some glitches were seen due to the shadows of the trees in the sides of the roads. The algorithm was simple and was effective in detecting the damaged road, and extracting the percentage of the damage that has occurred and it was able to take decision either to continue the same road or the next road.

5. FUTURE WORK

The current procedure is simple and basic. While certain improvements can be performed for real time applications and Geo tagging of the damaged areas so that the information can be shared with the road development boards to maintain the infrastructure better and safe for the transportation.

REFERENCES

1. Roberto Manduchi and James Coughlan, Computer Vision Without Sight, Communications of the ACM, pp. 96-104,Vol.55, no.1,Jan.2010.

2. Yanping Wang, Yanbin Wang, Yong Da, Xiaoyan Liu, Jiebo Li, Jingyi Huang. (2011). An Object-oriented Method for Road Damage Detection from High Resolution Remote Sensing Images. IEEE International Conference on Geoinformatics,

3. Lixia Gong, Liqiang An, Mingzhong Liu, Jingfa Zhang. (2012). Road damage detection from high-resolution RS image. IEEE International Conference on GeoScience and Remote Sensing.

4. Haijian Ma, Nan Lu, Linlin Ge, Qiang Li, Xinzhao You Xiaoxuan Li. (2013). Automatic Road Damage Detection Using High-Resolution Satellite Images And Road Maps. IEEE International Conference.

5. Xirini Zhang, Yan Chen, Mingquan Jia, Ling Tong, Youchun Lu, Yougxin Cao.(2013). The Study Of Road Damage Detection Based On High-Resolution SAR Image. IEEE International Conference.

6. Chenrong Fu, Yan Chen, Ling Tong, Mingquan Jia, Longfei Tan, Xiaonan Ji. (2014). Road Damage Information Extraction Using High Resolution SAR Imagery. IEEE International Conference.

7. Jianhua Wang, Qiming Qin, Jianghua Zhao, Xin Ye, Xuebin Qin, Xiucheng Yang, Jun Wang, Xiaopo Zheng, Yuejun Sun. (2015). A Knowledge-Based Method For Road Damage Detection Using High Resolution Remote Sensing Image. IEEE International Conference.

8. Xinzheng Yang, Yunjian Jiang, Yong Zhang, Zhiyuan Liu. (2015). Vehicle type and route selection for emergency logistic management under road damage. IEEE International Conference on Service Operations And Logistics, And Informatics (SOLI)

9. Li Qiang, Zhang Jingfa, Wang Na, Damaged Road Extraction From Post-Seismic Remote Sensing Images Based On Gis And Object- Oriented Method China Earthquake Administration, Harbin, China