Cost-Effective Vehicle Monitoring System for Detecting Unacceptable Driver Behaviors on Road

Cost-Effective Vehicle Monitoring System for Detecting Unacceptable Driver Behaviors on Road

Md. Sakibur Rahman Sajal

Department of Computer Science and Engineering United International University

Dhaka, Bangladesh

Dr. Salekul Islam

Maliha Jahan

Department of Information and Communication Technology

Bangladesh University of Professionals Dhaka, Bangladesh

Department of Computer Science and Engineering United International University

Dhaka, Bangladesh

Abstract Number of subscribers for private vehicle monitoring is increasing worldwide, especially in the developing countries because of the rapid growth of private cars and appointed drivers. GPS based car location and speed monitoring offers only a narrow window to observe the driver behavior on the road. Installation of cameras and other expensive sensors might not be feasible for all users and the collected data size will increase tremendously. To monitor the driving pattern of the drivers in the absence of the owner, we propose a cost-effective solution that is easily implementable with the help a smartphone or a cheap dedicated device with appreciable accuracy and versatile features. Application of an accelerometer sensor, together with the GPS module, provides the means to identify rough driving which includes hard brakes, fast accelerations, sharp turnings, over speeding on the roads. Besides these, the proposed algorithm also can identify reckless lane change, frequent acceleration-deceleration, crossing speed bumps without slowing down and especially unauthorized carriage of people or goods when the owner is not present. With supplementary sensors at a very little increase in cost, the proposed method can also be extended to monitor other concerns as like the proper use of indicators while turning, unnecessary use of air conditioning by wasting fuel and maintaining the right gear while driving etc.

Keywords Driver monitoring, GPS sensor, accelerometer, safe driving, lane change detection

1. INTRODUCTION

   In recent years, the number of private cars and their drivers have seen a tremendous increase, especially in the developing countries. As a result, car related incidents also increased which in most of the cases is because of reckless driving on the highways. Because of car owners tendency to appoint drivers instead of driving themselves, remote monitoring of cars condition and drivers behavior is therefore a much- needed service in developing countries where drivers are easily found. However, with the availability of cheap drivers, their unethical behavior has become very common now-a- days. Recently, the incident of drivers picking up unauthorized passengers without the car-owner knowledge is concerning everyone, especially who need their private cars to move about the city for picking and dropping family

   members at different destinations throughout the day. Another concern of the car-owners is whether the driver is driving safely and economically. Reckless lane change, frequent acceleration-deceleration and crossing road bumps without slowing down could end up in a fatal accident that will cost not only life but also take economic toll. Therefore, the research field of vehicle and driver behavior monitoring is certainly not a new one.

   There is quite a number of researches and studies relating to this field. While a portion of these works use in-vehicle sensors or external hardware and sensors, the other portion uses latest smartphone sensors. Most of the earlier works and some of the new works in this field falls on the first category. Ferreria et al. [1], in their paper, gathered driving data from 70 drivers using Driver Monitoring System to explore the effects of journey characteristics on driver behavior. Husni et al. [2] proposed a system to predict cars maintenance necessities and monitor cars functions using IoT. They claim that their system will increase cars efficiency and reduce fuel consumption. In [3], Wu et al. used motion sensor data and physical models to develop a novel driver behavior identification system. Howlett et al. investigated the feasibility of using Neural Network analysis on the time varying voltage wave form created from spark plug to perform fault diagnosis on engines [4]. All of them either used built-in sensors or installed dedicated devices for data collection.

   In [5] by Mohan et al., they presented Nericell, a system to monitor traffic conditions and road conditions using smartphone sensing components like microphone, accelerometer, GPS sensor, GSM radio etc. Singh et al. [10] also use smartphone inbuilt sensors to detect driving behavior. In [6], Fazeen et al. used inbuilt accelerometers to asses driver behavior and road conditions. They used a high- pass filter to filter the sensor readings and set thresholds for different driving incidents like acceleration, lane changing etc. Gazali et al. proposed a system to detect vehicle overtaking resulting in inconsistent driver behavior using

   available ready-made sensing technologies in [7]. In the work by Johnson et al. [8], they presented MIROAD, a system to detect different driving incidents. They used sensor fusion based on smartphone and Dynamic Time Warping method. In [11], Dai et al. compares drivers driving patterns collected using smartphone sensors with typical patterns of drunk driving for early detection of possible drunk driving. You et al. [12] present Carsafe, a driver safety Android application that uses machine Learning algorithms along with computer vision to detect drowsiness in drivers. The app uses phones front camera to detect if driver is distracted or drowsy and back camera to monitor road condition. In [13], Chen et al. propose a system to identify abnormal driving in real time with high accuracy. They studied six types of behaviors in driving, namely Swerving, Weaving, Fast U-turn, Sides lipping, Sudden braking and Wide turning, to recognize their unique patterns and applied to detect abnormal driving.

   A significant portion of related works in this field use camera footage to assess road condition and drivers condition. In [14], Li et al. uses deep Neural Network on data from inertial sensing components and camera videos to detect unusual driving maneuvers. Jo et al. [15] proposed a system to assess drivers drowsiness and distraction using video camera footage. Their system performs drowsiness detection when driver is looking ahead and distraction detection when driver is looking elsewhere. However, the trade-off is between employing cost-effective apparatus and the accuracy detail of the driving pattern detection. Specially, in the case of camera-based methods where either a smartphone or a separate camera hangs in front of the driver, facing toward him can cause driver discomfort. Besides incorporation of more sensors not only increase the installation cost but also make the data bulky for real-time processing. Especially for multi-sensor application, variety of data need lots of processing although the result is more reliable and accurate.

   To address these concerns, this paper proposes a system to remotely monitor cars conditions like, the number of passengers in the car, and driving behavior of driver, like, frequent lane-changing, hard braking, fast acceleration etc with the application of least of sensors i.e. accelerometer sensor combined with GPS module. The novelty of this work is to maximize the utility of accelerometer and GPS data to fully use it in versatile applications by extracting the embedded information which might not surface without proper processing. For the preliminary study purposes, we used smartphoneswith built-in accelerometer and GPS sensors since it will be carried by the driver himself thus eliminating any installation charge. However, we can easily convert the system to make a dedicated device with in a very little cost that can be installed in the vehicle to collect and transmit the data to the server continuously.

   The rest of the paper is organized as follows. Theory behind the proposed method is explained in section II. Materials and methods used for the study is described in details in III. Results of the experiments are presented in section IV. Finally, concluding remarks with the future direction are given in section V.

2. THEORY OF THE PROPOSED METHOD

   1. Alignmnet of the device with the vehicle

      Fig. 1. Data acquisition: a) Alignment of the device with the vehicles axes of co-oridinates, b) Acquired data from 3-axes accelerometer.

      For proper interpretation of the collected data, the device must be aligned with the cars axes of co-ordinates. As shown in Fig. 1 (a), a smartphone is placed on the dashboard of the car so that the lateral (side to side), longitudinal (back to front) and remaining upward (bottom to top) directions can be defined as the x, y and z-axis, respectively. The accelerations in these three directions are recorded as a time-series data and different incidents on the road can be identified from the analysis of these data (see Fig. 1 (b)) given that the directions were maintained throughout the data collection time. The GPS data will provide the locations of the incidents.

   2. x-axis data analysis: sharp turns and lane changes

      The x-axis data represents the acceleration along the lateral direction of the car (see Fig. 1(a)). Acceleration in this direction stems from couple of reasons. Firstly, the excessive centrifugal force due to sharp turns at road-bends gives a high acceleration value. Left turn or right turn can be identified from the polarity of the data about the zero line. The centrifugal acceleration, is quantified as

      (1)

      Here, v is the velocity, r is the radius of curvature and g is the acceleration due to gravity. Thus, both the velocity and the radius of curvature contributes to this acceleration. The maximum velocity for turning thus can be fixed at a given place if we set a threshold for maximum value of .

      The radius of curvature can be estimated from the GPS data (latitude and longitude converted in Cartesian co- ordinates) which helps us to set the maximum allowable velocity (in order to define a sharp turn as a function of curvature) while taking a turn. If we have three consecutive

      latitudes, and corresponding longitudes, , the radius of curvature, r of the turn can be calculated as

      (2)

      Where the parameters g, f and c are calculated form the following matrix inversion to solve the equation of a circle

      (3)

      Another source of centrifugal acceleration is the sharp/frequent lane changes while driving at a very high speed. Here too the polarity gives the direction of change (from left lane to right or vice versa). However, the lane change is reflected as a slow variation in the baseline of the acceleration data (see x-axis data in Fig. 1(b)) compared to the sharp turns.

      The high frequency vibratory noise in x-axis data is due to the unevenness of the road surface which can be eliminated before data analysis by wavelet filtering. The trend line then becomes visible and lane change/sharp turns detection can be performed with higher accuracy.

   3. y-axis data analysis: hard brakes and fast accelerations

      Data from y-axis can be analyzed for detecting hard brake or fast accelerations and also their frequency. Specially at the intersections or jam prone areas this sudden speeding or braking could raise the probability of collisions. Fast acceleration/deceleration can be detected by checking whether the rate of change of y-axis acceleration, exceeded a predefined threshold, as

      (4)

      In [16], this threshold is taken as 3m/s2 for detecting hard brakes or fast acceleration. However, this could be subjective and depended on the traffic density-based risk factors.

      Not only the event of fast accelerations and hard brakes but also the frequency of it is registered in the proposed system because this reckless behavior increase the possibility of accident. In such situation, the driver can be warned automatically by the system or manually by the owner who will be notified beforehand.

   4. z-axis data analysis: estimate of passenger count

      Analysis of the z-axis data can offer versatile applications. Firstly, the crossing of any speed bump (due to speed breaker or raised manholes) induces damped oscillation in the car which is recorded in the z-axis acceleration data [17]. The amplitude of the acceleration depends on the speed at which the car hits the bump which can be verified by the GPS data-based speed calculation. Also, the time period of the decaying oscillation is proportional to the mass of the car (which includes the mass of the passenger as well). The time period of any spring mass system is defined by

      (5)

      Therefore, as estimate of the number of passengers can be made from the frequency analysis of the z-axis acceleration data after it hits a bump. Also, it can be verified whether the car crossed the bump with a proper speed as well from the

      speed data at the event of hitting a speed bump. This will ensure the durability of the cars suspension mechanism [18].

   5. GPS data analysis

   Fig. 2. GPS data processing: a) Identification of the idle positons, b) Removal of the idle position data points for getting uniform spatial samples

   Before starting to analyze the acceleration data, in is required to remove the idle data points (where the car was stationary either for traffic signal or due to waiting at will). For that, we need to analyze the velocity of the car (see Fig. 2 (a)) and find the time instances where it was not moving. Thus, the time samples can easily be removed from the acceleration, GPS and velocity data as shown in Fig. 2 (b) to obtain spatial uniformity (see the reduction in number of total data samples).

   We need to analyze the GPS data for identifying the turn locations as well to distinguish sharp lane changes from fast turns. To do so, three consecutive GPS latitudes and longitudes are to be analyzed taking the middle sample as the origin (by subtracting its values from other two samples) as shown in Fig. 3. For any right or left turn, the product of slope m1 and m2 will be negative since there will an adjacent quadrant change. Otherwise the product will be positive. This will give the location (along with time) where the car took a turn.

   Fig. 3. Turn location identification using GPS data.

3. MATERIALS AND METHODS

   1. Data Collection

      Data was collected using a HUAWEI Y6 Pro 2019 Smartphone with built-in GPS and accelerometer sensor. The GPS data was captured by GPS Logger mobile application (developed by BasicAirData [http://www.basicairdata.eu]) and saved as a .txt file in the phone directory in which GPS based velocity is also provided by the application. The sampling frequency was 1Hz (maximum available resolution 1 second interval) for GPS data collection.

      The 3-axes accelerometer data was collected using Accelerometer mobile application developed by Keuwlsoft [https://www.keuwl.com/apps.html] with 50Hz sampling frequency in AC mode (alternating coupling) to eliminate DC offset due to phones position.

      As for the vehicle, TOYOTA Corolla AXIO (1400 kilogram) was used driven by a domestic driver (38 years of age with 11 years of driving experience). To minimize frequent braking because of pedestrian interfrence, data was collected during night time in a residential area. The track was travelled twice, first with NORMAL driving and later with AGGRESSIVE driving. The device was placed on the dashboard fastened by adhesive tape. All the data files were saved in the smartphone and then transferred to a Laptop. MATLAB 2019b was used for offline data processing, analysis and interpreting the result (driver behavior).

   2. Data Propressing

   Before analyzing the acceleration data, we used (Debauches) wavelet filtering of order 5 with level 4 (see Fig. 4 for z-axis data denoising) in order to remove high frequency noise. Fig. 4(a) shows the raw data after removing the idle locations and 4(b) shows the signal after removal of the high frequency noise in the z-axis data.

   Fig. 4. Denosing of the acceleratio data using wavelet filtering.

   The remaining intrinsic noise was eliminated by thresholding (with 0.5m/s2). Finally, the clean data is fed into different functions designed for specific purpose (see Fig. 5). Data from x-axis goes to a zero-crossing register for calculating the time interval between successive zero-crossing (see Fig. 5(a)) incident so that the duration of lane change and turning can be measured along with the acceleration value. Data from y-axis goes to a counter that records the number of occurrences of hard brakes and sharp acceleration based on

   [16] as shown in Fig. 5 (b). For this purpose, the time samples when the acceleration exceeds the threshold are represented as a delta spike with the magnitude representing the amount of excessive acceleration over the threshold (2m/s2).

   Fig. 5. Analysis of the 3-axes data for identification of unaceptable driver behavior. a) x-axis data: for turning duration and acceleration, b) y-axis data: for frequency of fast acc./ brakes and c) z-axis data: for oscillation freqency

   The remaining z-axis data is fed to a peak detector function to calculate the time period of the damped oscillation and also the magnitude of it. Thus, the information about the time periods (T1, T2 and T3 as shown in Fig. 5 (c)) are extracted to obtain the mass of the car using Eq. (5).

   The velocity of the car during the crossing of the speed bump or pot holes are readily available from the GPS based velocity data. The location of the speed bump can be registered based on a trigger function that detects high value of acceleration in the preprocessed z-axis data. Corresponding y-axis data can be utilized to assess whether the car was decelerating before hitting the bump (which would be the expected practice). Otherwise, we can record this reckless for future reference.

4. RESULT AND DISCUSSION

   1. Sharp turns and lane change identifications

      Fig. 6. Routes for collecting data. In the first trip, the driving was normal followed by an agreesive driving on the same road.

      Fig. 6 shows the route that was travelled for data collection. There were 7 turns to come back to the starting point (red dot) after driving normally, followed by another aggressive trip. Detection of turning events was verified by GPS data where the products of the slops in consecutive data points were negative. Otherwise, the occurring of a large acceleration along x-axis was contributed by sharp lane change.

      It should be remembered that due to the unevenness of the road, the car can generate acceleration along x-axis however the duration will be very short which can be filtered out with the help of GPS data (sampled at 1sec interval). Duration for at least 2sec was considered to be regarded as the occurrence of a lane change or take turn. Table 1 shows the result of data analysis from both of the normal an aggressive driving.

      Table -1 Experiment Result

      Detection of sharp turns			Number of sharp lane change
      Turn	Normal	Aggressive	Road	Normal	Aggressive
      1	×	×	1-2	0	2
      2	×		2-3	0	1
      3	×	×	3-4	1	3
      4			4-5	1	2
      5	×	×	5-6	0	2
      6	×		6-7	0	1
      7	×		7-0	0	3
      In %	14.28	57.14	Total	2	14

      For every turn if the acceleration crossed the threshold limit, we marked it with a , otherwise with a × in Table I. In datafile, we scored the detection with a 1 or a 0, respectively, in order to calculate a percentage for numeric evaluation. Similarly, the count of sharp lane changes on the road segments in between two turns are recorded along with the total count for the whole trip in both driving style. From the result, it is evident that the percentage of sharp turns and lane changes are quite high in aggressive driving compared to the normal driving. Although the result will differ if the threshold values changed based on traffic condition and road quality.

   2. Occurance of hard bakes and fast acceleration

      Detection of hard brakes or fast accelerations is very straight forward. The time instances when the car exceeded the predefined thresholds are recorded with the amplitude of excessing acceleration over the limit. It is done to get a weighted score which will not only reflect the frequency of the occurrence but also the severity of them. Therefore, the score is given as average excess acceleration with occurrence frequency. Score for the normal driving was found to be 0.2 with frequency of 2 while that for the aggressive driving was 0.45 with a frequency of 6.

   3. Estimation of passenger count

   Before estimating number of passengers from the time period of the oscillation of the car, we had to calibrate it for the specific car. Therefore, the fundamental frequency of the oscillation with the driver inside was calculated from the z- axis acceleration data. Then using Eq. (5) with the known value of time period, T and combined mass (car plus the driver), m we calculated the spring constant of the cars suspension. A theoretical line is then plotted as shown in Fig. 7 for an increment of mass by 55kg (average). To verify the theory, we then increased the number of passengers one at a time who had average weight of 56.6kg with standard deviation of 7.8kg. Each time the same speed bump was crossed maintaining the same velocity and the time period was calculated from the fundamental frequency. As seen from the figure, the practical time period matched the theoretical line with 0.8%. With the help of the time period ranges, number of passengers was estimated accurately. However, the accuracy will if the mean passenger weight does not lie in the vicinity of the average mass considered for deriving the calibration line.

   It is to be remembered that the oscillation frequency does not depend on the velocity of the car while hitting a bump, however the amplitude of the acceleration depends on it. Therefore, by detecting the occurrence of a high value of acceleration along z-axis it can be easily determined whether the car driver detected the road bump in time and started to decelerate or ignored it completely with the help of y-axis data. In our study, we slowed down in front of the bump for both driving mode for the cars safety.

   Fig. 7. Estimation of the number of passengers from the time period of oscillation of he car after hitting a raod bump.

5. CONCLUSION AND FUTURE WORK

In this paper, we described a cost-effective system for car owners to monitor drivers driving behavior and car conditions remotely using least of sensor to reduce data size without compromising the accuracy. Our system can estimate the number of people inside the car in real time and assess drivers driving behavior by detecting lane changing and hard braking or fast acceleration incidents using accelerometer data from smartphones. The GPS data not only gives the velocity and the location of the car but also can distinguish between closely related events. Thus, we have utilized the information embedded in these two data to a greater extent. Our system is cost effective as no additional hardware or sensors are needed, and it has a high potential to be used in a wide scope of applications. Car owners can use our system to ensure their drivers are not picking up passengers without them knowing. They can also check if the driver is driving recklessly in the absence of the car owners. Industries and companies can use our system for efficient fleet management. Our system can be used to implement a large-scale system for automated traffic rule violation detection. We are now working on expanding the scope of driving behavior assessment by detecting if the driver is driving in the wrong side of the road. This will be included in our future work. For our future work, we will also include the estimation of fuel consumption by a vehicle which will encourage drivers to drive economically and save fuel thus provide a complete solution of this pressing concern.

REFERENCES

1. S. Ferreira, Z. Kokkinogenis, and A. Couto, Using real-life alertbased data to analyse drowsiness and distraction of commercial drivers, Transp. Res. F, Traffic Psychol. Behav., vol. 60, pp. 2536, Jan. 2019.

2. E. Husni, G. B. Hertantyo, D. W. Wicaksono, F. C. Hasibuan, A. U. Rahayu and M. A. Triawan, "Applied Internet of Things (IoT): Car monitoring system using IBM BlueMix," 2016 International Seminar on Intelligent Technology and Its Applications (ISITIA), Lombok, 2016, pp.417-422.doi:10.1109/ISITIA.2016.7828696

3. M Wu, S. Zhang, and Y. Dong, A novel model-based driving behavior recogniion system using motion sensors, Sensors, vol. 16, no. 10, p. 1746, Oct. 2016.

4. Hwlett, R.J.: 'Condition monitoring and fault diagnosis in a domestic car engine using a neural network', IET Conference Proceedings, 1996, p. 5-5, DOI: 10.1049/ic:19961142

5. P Mohan, V.N. Padmanabhan, R. Ramjee (2008) Nericell: rich monitorng of road and traffic conditions usingmobile smartphones, in Proceedings of the 6th ACMConference on Embedded Network Sensor Systems – SenSys 08 (2008), p. 323.

   https://doi.org/10.1145/1460412.1460444

6. M. Fazeen, B. Gozick, R. Dantu, M. Bhukhiya, M.C. González, Safe driving using mobile phones. IEEE Trans. Intell. Transp. Syst. 13(3), 14621468 (2012). https://doi.org/10.1109/TITS.2012.2187640

7. E.H. Gazali, Monitoring Erratic Driving Behaviour caused by Vehicle Overtaking using Offthe-shelf Technologies. October (October) (2010)

8. D.A. Johnson, M.M. Trivedi, Driving style recognition using a smartphone as a sensor platform, in Proceedings of the IEEE Conference on Intelligent Transportation Systems, ITSC, pp. 1609 1615 (2011). https://doi.org/10.1109/ITSC.2011.6083078

9. X. Li, W. Wang, M. Roetting. (2018). Estimating Driver's Lane- Change Intent Considering Driving Style and Contextual Traffic. IEEE Transactions on Intelligent Transportation Systems. PP. 10.1109/TITS.2018.2873595.

10. P. Singh, N. Juneja, S. Kapoor, Using mobile phone sensors to detect driving behavior, in Proceedings of the 3rd ACM Symposium on Computing for Development – ACM DEV 13 (2013),p. 1. https://doi.org/10.1145/2442882.2442941,

11. J. Dai, J. Teng, X. Bai, Mobile phone based drunk driving detection., in 2010 4th International (2010), pp. 18.

    https://doi.org/10.4108/ICST.PERVASIVEHEALTH2010.8901,

    http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5482295

12. C.-W. You et al., CarSafe app: Alerting drowsy and distracted drivers using dual cameras on smartphones, in Proc. 11th Annu. Int. Conf. Mobile Syst., Appl., Services, Jun. 2013, pp. 1326.

13. Z. Chen, J. Yu, Y. Zhu, Y. Chen, and M. Li, D3: Abnormal driving behaviors detection and identification using smartphone sensors, in Proc. 12th Annu. IEEE Int. Conf. Sens., Commun., Netw. (SECON), Jun. 2015, pp. 524532.

14. H. Li, H. Wang, L. Liu, and M. Gruteser, Automatic unusual driving event identification for dependable self-driving, in Proc. 16th ACM Conf. Embedded Netw. Sensor Syst., New York, NY, USA, Nov. 2018, pp. 1527.

15. Jo, S. J. Lee, J. Kim, H. G. Jung, and K. R. Park, Vision-based method for detecting driver drowsiness and distraction in driver monitoring system, Proc. SPIE, vol. 50, no. 12, Dec. 2011, Art. no. 127202.

16. A. Beinstein and T. Sumers, How Uber Engineering Increases Safe Driving with Telematics UBER Engineering, 2016, URL: https://eng.uber.com/telematics/

17. Measuring Automobile Spring Constant,

    url: https://mcanv.com/Answers/qa_masc.html

18. D. Garcia-Pozuelo, A. Gauchia, E. Olmeda and V. Diaz, Bump Modeling and Vehicle Vertical Dynamics Prediction , Advances in Mechanical Engineering, 2015, doi: https://doi.org/10.1155/2014/736576