Accident Detection, Severity Prediction, Identification of Accident Prone Areas in India and Feasibility Study using Improved Image Segmentation, Machine Learning and Sensors

Accident Detection, Severity Prediction, Identification of Accident Prone Areas in India and Feasibility Study using Improved Image Segmentation, Machine Learning and Sensors

Vipul Gaurav* Department of Computer Science, Sir M. Visvesvaraya Institute of

Technology, Bengaluru, India

Sanyam Kumar Singh* Department of Information Science, Sir M. Visvesvaraya Institute of Technology, Bengaluru, India,

Avikant Srivastava*

Department of Information Science, Sir M. Visvesvaraya Institute of Technology, Bengaluru, India,

Abstract This paper provides an insight into a complete solution for accident detection and severity prediction with an alert system using machine learning and computer vision. It identifies the accident-prone areas across India for the feasibility study of the system and makes use of image segmentation using masked recurrent neural networks to detect road accidents proposed as an improved segmentation over its counterpart processing techniques. The use of piezoelectric shock sensors validates the prediction of the accident and a mitigation alert system is designed to inform the concerned authorities regarding the occurrence of severe road accidents. The system also studies the severity of the accidents which can be used to analyze its impact economically. The solution proves to be an effective tool for accident detection, prediction, alert and severity analysis system for states across India.

Keywords Accident detection, machine learning, computer vision, image segmentation, masked recurrent neural networks, piezoelectric shock sensors, alert system

1. INTRODUCTION

   Road accidents in India are a major cause of decreasing life expectancy with road accidents contributing to over 148,000 deaths [1] out of 467,000 deaths in 2016. Indian Economy has a hit of 3 percent of GDP growth due to road accidents as per the United Nations with an estimated loss of $58,000 in terms of value every year. The metropolitan cities such as Chennai, Mumbai and New Delhi have been increasingly highlighted for lack of road safety and rash driving cases. The recent trends show that there has been an increase in the global number of road accidents even in developed countries [2]. However, under-developed and developing countries suffer a more significant impact due to life and economic losses. These accidents occur due to violation of traffic safety rules, careless rash driving, driver drowsiness and lack of good quality roads. The problem becomes more adverse for highways and hilly areas where accidents are unavoidable. Road accidents are characterized by high death rates due to delay in arrival of help and inefficient systems of mitigation to alert the concerned authorities. Road accidents on the highways are typically caused by natural reasons such as extreme weather conditions such as fog and consecutive collision of vehicles are common on Indian

   highways due to lack of visibility. The states of Maharashtra, Tamil Nadu and Uttar Pradesh account for the highest number of road accidents in India [3]. The problem can be handled by making use of computer vision and low-cost sensor networks. The current solutions involved heavy dependency on sensor networks and area coverage. This can be substantially replaced by making use of object detection and image segmentation for accident classification. The system identifies the accident-prone areas which are the target stakeholders for the deployment and sets it apart from other implementations since it provides a feasibility factor associated with it. Furthermore, the system provides enhanced mitigation alert to the concerned authorities which helps in preventing any consecutive collisions that could possibly lead to greater loss of lives [4].

2. LITERATURE SURVEY

   Nicky Kuttukkaran et al. [4] proposed heartbeat a major factor for differentiating accidents then preferred Bluetooth for sending alert signals. This approach is limited by the range of Bluetooth and heartbeat is not a suitable measurement for accident verification because it is subjective to interpretation.

   Sanjay Kumar Singh [5] studied road accidents in India at state, city and rural levels to find out that 50% of the cities are prone to road accident fatalities and the number of accidents is estimated to rise to 250,000 by 2025. Chaitali Khandbahale et al. [6] performed multi-object detection using embedded detection for accident prevention. This approach has the limitation of high economic cost and the embedded system comprises sensors that may not function properly during rainy forecasts. Nedjet Dogru et al. [7] applied a clustering algorithm for grouping vehicles based on speed and location. This perspective will only give data about the pattern of movement and the model did not provide optimum accuracy at the time of deployment. Vipul Rana et. al [8] proposed accidental prevention through output and user side experience by asking users to submit a form about the type of climate, type of vehicles, etc. This verification is limited to those cases where the victim can fill the form. Deeksha Gour et al. [9] proposed optimized YOLO working on a system for accident detection. This model has the

   possibility of misclassification of the required target object that can lower the efficiency of the system. Xi Jianfeng et al.[10] adopted support vector machine (SVM) for accident detection which is found to be limited while localizing objects and instance segmentation during detection hence instance classification is hindered. Raad Ahmed Hadi specifically studied the use of background subtraction, feature-based tracking and region-based tracking methods in the application of vehicle detection [11]. Zhong-Qiu Zhao et al. [12] proposed generic object detection to classify existing objects in an image using masking and recurrent neural networks. This method reduces preprocessing time as well as enhances the pipeline for detection and validation and it can further be customized for vehicle detection. Kaiming He et al. [13] extended fast R-CNN using masking to perform instance segmentation that can be potentially used for custom object detection applications including car accidents. Sodikov et al. [14] studied the accidents occurring in Uzbekistan to provide a visualization of the traffic accident distribution with respect to the day, week, month and year from which we can draw insights about the accident frequency and severity of the road accidents in any country in general. S. Ramya et al. [15] studied various parameters affecting road accident severity and found out that Random Forest Classifier outperforms its counterparts in handling accident severity prediction due to its ability to handle seasonality in accidental vehicles and casualties. Jabar H. Yousif et al. [16] presented an approach to construct an optimal neural network for traffic accident severity prediction with a proper feature selection to handle any correlations between the severity of accidents and weather conditions in Jordan and can be extended to other countries as well.

3. METHODOLOGY

   The system provides a three-phase solution to analyze road accidents in India using machine learning and computer vision. The block diagram covers the whole approach in a broad sense. The central component of the block diagram represents the selected approach whereas its counterparts indicate the experimented techniques. After testing the accuracy and results of all the methods. We came up with an optimal approach for each and every task.

   Figure 1: Accident Detection and Severity Prediction Approach

   The first phase includes the identification of accident- prone aeas in India based on the number of accidents occurring across various states and union territories. For this, we made use of the data provided by the Government of India which consists of year-wise historical data from 2003 to 2016 that can be used to predict the number of accidents that can occur in different states in the upcoming future. This analysis aimed to identify accident-prone areas where the model would be feasible to deploy among all the states across India. Accident-prone zones are the areas that have experienced the highest frequency of road accidents due to various factors and are the primary stakeholders for this project. The data provided by the Government of India was highly linear in terms of variance and scattered across as a continuous distribution. Owing to this, we applied various regression techniques such as linear regression, elastic regression, and ridge regression. Out of these, linear regression handled the continuous data promptly and outperformed its counterparts giving a much appreciable accuracy of 95%.

   Figure 2: Year-wise number of accidents in India

   The second phase of the solution involves the classification of car accidents. This phase can be implemented by using any image segmentation and object detection algorithm. We experimented with You Look Only Once (YOLO) algorithm as suggested as a classic object detection approach [9]. However, the probability of misclassification of an accident is high and this becomes a significant problem when employed with real-time data. Image segmentation is an alternative approach to classify car accidents. We experimented with the watershed algorithm, canny edge detection and auto canny algorithms for segregation of the cars and accidental cars. The limitation of the YOLO algorithm found was that the region of interest was satisfactory but it misclassified as well as missed some of the cars as provided in the input video footage. The accuracy wasnt feasible for the deployment of the given algorithm for the model. Hence, we explored alternative approaches to provide better segmentation results as well as techniques to preprocess the input data.

   Figure 3: YOLO output on vehicle detection

   Figure 4: Watershed Segmentation of Car Accident

   Figure 5: Auto Canny Edge Detection of Car Accident

   Thus, we applied watershed segmentation to the video footage and images of car accidents available from the data provided by the Government of India. The major drawback of the watershed segmentation was over-segmentation of the image where it learns the gradients more than the context needed and hence makes it prone to overfitting on a large number of images and video footage. Canny edge detection performs optimally well for the car accident classification and has a keen region of interest which makes it suitable for a good preprocessing technique for image augmentation of accidental car images. The problem with most of the widespread architectures used for object detection is that they perform semantic segmentation which cannot differentiate between two cars belonging to the same class. Thus, the need arises for image segmentation based on instance segmentation where we need to differentiate between the cars of the same brand as well in an accident. For this, we know that object instance segmentation is carried out effectively by using recurrent neural networks[12]. This technique helps us to identify the cars in an accident effectively as separate instances and gives much better accuracy of up to 92%.

   Figure 6: RCNN output without preprocessing

   Figure 7: RCNN output on Watershed Segmentation Result

   In case of some car accidents where the image components are bright enough, we do not need to apply any image augmentation or segmentation algorithm and the R-

   CNN can detect a proper region of interest without much depreciation in terms of output quality. To further enhance the segmentation, we make use of masking on top of Fast R- CNN [13]. Fast R-CNN provided us with proper identification of the region of interest and enables us to perform semantic segmentation where the objects belonging to the same class are classified equally. However, for studying the severity of an accident we need to be able to differentiate between the objects and cars at a granular level which can be achieved by using masked R-CNN that can identify boundaries and reform the region of interest to contain only the accidental portion of the cars which makes it superior to other approaches for image segmentation in case of car accidents.

   Figure 8: RCNN output without preprocessing for brighter image

   After image segmentation accidents are divided into three categories based on their severity. This is done through severity analysis of the accidents. The accidents are clustered into multiple classes based on the degree of damage ranging from low, intermediate to high. Followed by identifying multiple classes various experimental techniques are applied to obtain the classification of accident severity. Road accidents involved various factors including the number of vehicles involved in the accident and the number of casualties. The police force arriving at the scene and the circumstances in the form of road conditions, weather and lighting involved in the area. The data is biased towards urban areas and the speed limit was highly correlated with urban areas indicating a frequent violation of traffic rules and rash driving that contributed to a higher frequency of accidents. The accidents are classified as low, intermediate and high severity based on these parameters. For this purpose, we experimented with Logistic Regression and Gradient Boosting Classifier [16] which yielded partial success not enough for feasible deployment. However, Random Forest Classifier gave accurate results up to 92% and handled the correlations involved with the data with ease. The seasonal changes in the data in terms of different number of accidents caused by extreme weather conditions were highly correlated with the road surface and lighting conditions that were handled well with Random Forest Classifier since it acts as an ensemble classifier giving better decisions.

   Table 1: Parameters for Accident Severity Prediction

   The third phase of the solution involves the designing of a mitigation alert system. Based on the accident-prone areas identified, the device with cameras and CCTV for data collection can be employed. This would automate handling of accidental hazards on the accident-prone highways. In the first step, we would be using the mask R-CNN designed to analyze CCTV footage. There will be strips of piezoelectric sensors in the accident-prone zones. It will be a two-step validation since apart from object detection, piezoelectric sensors will detect the weight of the object. Hence the accuracy would be further increased. The potential false positives which could occur due to misclassification by the R-CNN are well handled with this approach. Alert messages will be sent to the concerned authorities through a cloud communications platform, completely integrable and programmable for sending and receiving data. Alert would only be given when there is similarity in the validated outputs of both object detection module and piezoelectric shock sensors. The model will categories accidents into three different types based on the intensity of the accident through an RCNN model. This RCNN model will first do semantic segmentation of the accident class followed by instance segmentation to separate all the accidents with each other based on severity analysis. Accidents with high impact intensity would be segregated into high priority alerts whereas the accidents with low impact intensity would be segregated into low priority alerts. This will eliminate the chances of ignoring serious accidents while handling less intense accidents. Thus, the approach is optimized with object detection and validation in the three-phased solution.

4. RESULTS AND DISCUSSION

   The system provides a complete solution to identifying, deteting and mitigating about accidents and also provides the feasibility study to identify accident-prone areas across India. The linear regression model outperformed its counterparts to predict the number of potential accidents that can occur in any given state across India with an accuracy of 95%. This helped us identify Maharashtra, Tamil Nadu and Uttar Pradesh as the most vulnerable road accident zones which makes them the target regions of interest to deploy this model in India.

   with different severity of accidents and the police stations that can be contacted immediately. Through a coordinated effort between the device and the authorities, a monitoring system can be developed which can lower down the loss of lives due to accidents to a large degree. A tracking system can also be developed for exclusive monitoring of accident- prone areas and link it with the developed system for optimal and efficient accident prediction to curb this problem in the near future.

   Figure 9: Number of Accidents occurring across states of India

   The image segmentation of the accidents is handled by the masked R-CNN model which provides accurate instance segmentation better than the Fast R-CNN model that can provide only semantic segmentation and cannot differentiate between the classes. We experimented with various image augmentation techniques and found out watershed segmentation and canny edge detection to be suitable for identifying the region of interest of any car accident images or footage. This enhanced the segmentation result given by the masked R-CNN taking edges into account from the region of interest identified from the input data.

   Figure 10: Final Result of Instance Segmentation with Masked R-CNN

   We also made use of piezoelectric shock sensors in highly prone areas where the frequency of accidents is extremely high such as Mumbai-Pune Highway to detect the shock generated during a collision and this acts as an extra validation step in predicting the occurrence of road accidents with high severity. The alert system is based on the outputs of the image segmentation, piezoelectric sensors and the accident severity classification model which is integrable with any monitoring device. The accident severity prediction involved multi-class classification with a range from 1 to 3 indicating the higher severity of accidents respectively and Random Forest Classifier outperformed the other counterparts in determining the severity of any road accident up to an accuracy of 92% which is feasible enough for deployment.

5. CONCLUSION AND FUTURE SCOPE

The paper provides an insight for the design of an alert system to mitigate concerned authorities regarding road accidents. This can be enhanced by through a data collection system that can keep a record of the nearest hospitals dealing

REFERENCES

1. National Crime Records Bureau of Data, India, 2016

2. The World Health Organization Global Status Report on Road Safety,

   The World Health Organization; Geneva, Switzerland, 2018

3. Manisha Nuikar, National Statistics of Road Accidents in India, Journal of Orthopaedics, Traumatology and Rehabilitation, January, 2013

4. Nikki Kattukkaran, Arun George, Mithun Haridas T.P., Intelligent Accident Detection and Alert System for Emergency Medical Assistance, IEEE International Conference on Computer Communication and Informatics, 2017

5. Sanjay Kumar Singh, Road Traffic Accidents in India: Issues and Challenges, World Conference on Transport Research – WCTR 2016 Shanghai. 10-15 July, 2016.

6. Chaitali Khandbahale, Nikita Pawar, Jay Rathod, Karan More, Multi Object Detection in Vehicle for Accident Prevention and Safety Assistance using Embedded System – Implementation Paper, International Journal of Engineering Research & Technology, May 5, 2019.

7. Nedjet Dogru,Abdulhamit Subasi, Traffic Accident Detection By Using Machine Learning Methods, International Symposium on Sustainable Development , March 2012

8. Vipul Rana, Hemant Joshi, Deepak Parmar, Pradnya Jadhav Monika Kanojiya, Accident Detection using Machine Learning Algorithm, International Research Journal of Engineering and Technology(IRJET), March 3, 2019

9. Deeksha Gour , Amit Kanskar, Optimised YOLO:Algorithm for CPU to Detect Road Traffic Accident and Alert System, International Journal of Engineering Research & Technology (IJERT), September 16, 2019

10. Xi Jianfeng, Guo Hongyu, Tian Jian, Lisa Liu, Liu Haizhu, A Classification and Recognition Model for the Severity of Road Traffic Accidents , SAGE Journal , May 22 ,2019

11. Raad Ahmed Hadi, Ghazali Sulong, Loay Edwar George, Vehicle Detection and Tracking Techniques: A Concise Review, Signal and Image Processing: an International Journal, Vol 5, February, 2014

12. Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu, Object Detection with Deep Learning: A Review, IEEE Transactions on Neural Networks and Learning Systems, April 16, 2019

13. Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross Girshick, Mask R-CNN, Facebook AI Research (FAIR), 2018

14. Sodikov, Jamshid, Road Traffic Accident Data Analysis and Visualization in R, International Journal of Computer Science Engineering and Information Technology Research (IJCSEITR), 2018

15. S. Ramya, SK Reshma, V. Dhatri Manogana, Y. Satya Saroja, Accident Severity Prediction using Data Mining Methods, International Journal of Scientific Research in Computer Science,

    Engineering and Information Technology, Vol 5, Issue 2, 2019

16. Jabar H. Yousif, Al Rababba Mamoun S, Neural Technique for Predicting Traffic Accidents in Jordan, Journal of American Sciences, November, 2013