Driver Drowsiness and Movement Detection based Vehicle Safety System

Driver Drowsiness and Movement Detection based Vehicle Safety System

Liya Anna Sebastian

Dept. of Comp Science &Engineering Mangalam College of Engineering,

Kottayam, India

Sneha Roy

Dept. of Comp Science & Engineering Mangalam College of Engineering, Kottayam, India

Vineetha V Nair

Dept. of Comp Science & Engineering Mangalam College of Engineering, Kottayam, India

Sona M S

Dept. of Comp Science & Engineering Mangalam College of Engineering,

Kottayam, India

Soniya Babu

Dept. of Comp Science &Engineering Mangalam College of Engineering, Kottayam, India

AbstractAccording to the National Highway Traffic Safety Administration (NHTSA) and the World Health Organization (WHO), over 100,000 people die in car accidents each year. Drowsy driving is a prominent cause of deadly road accidents all over the world. The goal of this research was to develop a road accident prevention system that detects driver tiredness, estimates driving inattention, and issues a warning when an oncoming hazard occurs. Drowsiness, on the other hand, is a natural occurrence in the human body that occurs owing to a variety of circumstances. One of the drowsiness parameters is the frequency of eye blinks. For preventing accidents, in drowsiness, we use computer vision to identify eye blinks. And we can detect a driver's other activities while driving by detecting his body movements. If a person is not concentrating on driving or making normal movements while driving, it will reflect in his driving quality, so with the help of a set of pressure sensors, we can detect the movements and provide proper alerts and warnings to the person.

KeywordsOpenCV, IoT, pressure sensors, potentiometer, firebase

I.INTRODUCTION

According to the National Safety Council, 30% of drivers admitted that they feel tired and sleepy when they are driving. Almost 4% of them resulted in accidents and are responsible for 80% of near-miss accidents. Driver drowsiness is the top cause of traffic accidents. In the realm of accident-avoidance systems, developing an expert system for detecting fatigued drivers has become a critical demand and problem. We provide a fully automatic solution to the problem of drowsy driving detection in this paper. There are several technologies for detecting drowsiness which can be classified into three categories 1) driver eye monitoring 2) steering pattern monitoring 3) psychological measurement. All the monitoring and measurement pieces of information are stored in the firebase. Our system uses a webcam and detects the pattern of eyelid closure for a long duration and it only focuses on eye facial landmarks without detecting drivers yawning. The duration of eyeblink is the time spent with the upper and lower highlights connected. It determines whether

the eyes are in an open position or closed using computer vision. Also in the system, we check the body movement of the driver with a set of pressure sensors installed in the drivers seat. We measure steering pattern deviation with the help of a potentiometer. When drowsiness detection happens, the system produces an alert sound and sends a message to the driver which is displayed on the screen. The goal of this research is to identify tiredness in drivers, which will help to prevent accidents and increase highway safety.

II.METHODOLOGY

An IoT-based total system is designed to keep away from endless mishaps due to driver drowsiness and psychological changes by focusing on eye movements. Computer vision helps to track and analyze the ratio of eye blinking and thereby providing a vigilant alert. Apart from it, the seating, as well as steering patterns against orderly behavior, can be treated as divergent practice and a warning is employed using LCD or buzzers.

SYSTEM ARCHITECTURE

Our model ensures an intelligent or smart vehicle system through advanced technology. This paper implements a mechanism to alert the driver on the condition of drowsiness or based on some physical features. A camera monitors the drivers eye blinking, eye closure, face detection, head posture, etc. with a face landmark algorithm and Euclidean distance in the behavioral-based approach. The eye region of the driver is extracted to determine the eye closure ratio. If this ratio deteriorates from the standard ratio, an alert is passed using a buzzer and LCD to display the warning. Other factors which can cause accidents are harsh dealing with the steering as well as driver posture in the seat. Deviation from the normal behavioral or physical pattern of the driver can be treated as risky. The pressure sensors are fine to handle the alert system based on seating positions. Highly qualified three pressure sensors were placed, one on the seat, another one on the leaning seat, and finally on the headrest. The sensors satisfactorily capture the physical appearance of the

driver to ensure that he sits straight and concentrates on the driving. If the sensors detect any abnormal behavior, the message can be informed to the driver through the buzzer sound and a display. Another exciting feature of our collab model is the usage of a potentiometer to analyze the steering pattern. If the driver turns the steering very fastly in a wide degree position, a devastating result might happen. In this case, also, a warning can be made through our hardware components. The alert that the system produces will be very precise which will ensure the driver gets a message on what part he/she has to be rectified. Different audio and text signals will be given according to varied situations. A switch is attached to the system and the alert message will be stopped only when the driver turns off the switch. Thus the driver can be made conscious. Firebase stores overall results which can be fetched when needed.

fig 1:

1. Driver Drowsiness Detection

   The proposed model is based on the flow chart in Fig-1, which shows how it works. The camera is turning on and capturing a succession of photographs, one after the other. It is determined whether the eyelids are open or closed. The driver of the vehicle is given a warning.

   fig 2:

   1. Face detection and localization

      Input can be captured using the webcam. To discover the face in the image, first convert the image into grayscale as the OpenCV algorithm for object detection takes gray images as

      the input. The method cv2.VideoCapture() is used to access the camera to grab the image. A Color image is not strictly necessary to detect the image.

   2. Facial landmark and extraction of the eye region

      In the model, we are using facial landmark prediction for the detection of eye regions. Facial landmarks are used to represent and identify important regions of the face, such as the Eyes, Eyebrows, Nose, Mouth, and Jawline. Facial landmarks have been successfully applied for face alignment, head position estimation, eye blinking detection, and much more. The goal of using facial landmarks is to discover salient facial structures on the face using shape prediction methods. Dlib is utilized to extract features from the face and predict the landmark using its pre-trained face landmark detector.

      The method includes:

      Here we used a training set of labeled facial landmarks on an image. These images are labeled, specifying specific (x, y) coordinates of regions surrounding each facial structure.

      The pre-trained facial landmark inside the dlib library is used to estimate the location of 68 (x,y) coordinates that map the facial structure and shape of the face. From the facial landmarks, we extract ee position and states.

      The indexes of the 68 coordinates can be visualized in the image below:

      fig 3:Visualizing the 68 facial landmark coordinates

   3. Eye Aspect Ratio

      The eye blink is fast closing and reopening of the eye. Each individual has a little bit of a different pattern of blinks. The pattern mainly varies in the speed of closing and opening and duration. The eye landmarks are detected. Each eye is represented by using 6 (x, y) coordinates, that is starting from the left corner of the eye, and then clockwise around the remainder of the region. There is a relationship between the height and width of these coordinates. The equation which reflects this relation (EAR) is given by:

      Where x1,….,x6 are 2D facial landmark locations.

      The distance between the eyelids is calculated as follows:

      When the Euclidean distance between and is given by ED(X, Y); these are two Cartesian coordinate points. As a result, it is represented in the Python program below:

      The numerator of this equation evaluates the distance between the vertical eye landmarks while the denominator evaluates the distance between horizontal eye landmarks. The eye aspect ratio (EAR) is constant ie; indicating the eye is open, then rapidly drops to zero, then increases again, indicating a blink has taken place. We are monitoring the eye aspect ratio to understand if the value falls but does not increase again, thus indicating that the person has closed their eyes.

      fig 4: Open and closed eyes landmarks

      fig 5: EAR for a single blink

   4. Calculation of the score

   A score is essentially a number that will be used to measure how long the person has closed his eyes and the rate at which they blink. The decision for the eye state is made based on EAR calculated using observed eye landmark coordinate values. So, if both eyes are closed, we will keep on increasing the score and when the eyes are open, we decrease the score. A threshold value is set to focus on the rate of eye blinking as well as the duration. If the eye aspect ratio falls below this threshold, well start counting the number of frames the person has closed their eyes and our system gets an expected warning.

2. Analysis of Steering Wheel Motion

   In general, driving task characteristics (e.g. speed, curvature, and lane width), driver features (e.g. driving experience), and driver states influence steering behavior (e.g. laxness, distraction, or fatigue). Drivers are continually assessing the environment ahead of them and making small, smooth steering adjustments to compensate for minor road bumps

   and crosswinds. When drivers become tired, their steering behavior can be affected.

   Rash driving can lead to road accidents. The suggested method is entirely IoT-based. The goal of this system is to reduce rash driving-related accidents on our streets for the sake of safety. Rash driving can be defined as a quick shift in speed or a continual change in path, among other things. The system monitors the driving pattern of the driver using a potentiometer and when the vehicle abruptly changes direction or accelerates, the system identifies this and flags it as reckless driving. The system then sends an alert message to the owner of the vehicle.

   fig 6a: Drifting

   fig 6b: Over Speeding

   fig 6c: Sharp Turn

   Drifting, overspeeding, and sharp turns can be considered rash driving. Drifting occurs when the steering wheel is rotated in a specific direction. When the variation exceeds the edge, a weird curved development of the vehicle is observed, indicating that the driver is having difficulty maintaining the path position. In rash driving examples, a common assumption is that the driver isn't maintaining a regular speed and is overspeeding while driving, causing him to drive unevenly. This is a situation where a vehicle is allowed or forced to turn past its planned restraint. If someone is driving

   at a high speed and suddenly yanks the steering wheel with a severe input, the car will jerk abruptly into a fast turn. The system will recognize this as rash driving.

   The steering wheel of a vehicle can rotate 360 degrees. The ratio is 360:24 = 15:1 if one full 360-degree turn of the steering wheel causes the wheels to turn 24 degrees. A potentiometer can also be turned 360 degrees. If the steering wheel is at 0 degrees, then the resistance will also be 0. As it rotates, the resistance also increases. When the resistance increases, a voltage drop is created. The position of the steering can be calculated using the ADC value corresponding to this voltage. The changes in the steering wheel are monitored and if the value obtained is unsafe, then an alert message is given to the driver. Because of its low cost, the proposed technology could assist to spread the usage of individual drowsiness detection in the real world.

3. Detection of Driver Posture Change

   Distracted driving is currently one of the most common causes of traffic accidents. Driver distraction causes many crashes, and a real-time distraction warning system can reduce or avoid many of them. As a result, keeping an eye on driver attention is being explored as a way to improve traffic safety. Driver postural data is useful for understanding the driver's intent and seating posture, and pressure sensors are a good choice for capturing it. The driver's motions, such as trunk rotation and foot operation, can reveal important details regarding the driver's physical condition and intent.

   fig 8:

   Using pressure monitoring, we suggested a method for detecting changes in driving position. Findings demonstrated that a driver's posture alteration could involve various body regions. The pressure sensors are placed in three different regions of the seat. Three sensors are used for this which are fixed in the headrest, backrest, and seat pan of the seat after demonstrating that the sensor could detect variations in mechanical load. The sensor was able to track the motions of the driver. When a person leans left or right with their upper body and is not facing straight while driving, two sensors fitted to the headrest and backrest of a chair could detect it. The back cushion and head support, as shown in Fig., are the most informative areas where the pressure difference is more pronounced. The pressure fluctuations in the sensors are tracked. The sensor with the greatest variation is picked to determine the driver's movement. As a result, the system can determine which body part is moving. The alert message appears on the LCD display to notify the driver.

   Distracted driving causes

   fig 7:

   • Speaking with someone on the phone.

   • Texting and other forms of phone tampering

   • Having a conversation with another traveler.

   • In the vehicle, there are moving objects/animals.

   • Changing the audio and/or climate settings.

   • Manipulation of vehicle controls and components.

   • Reaching for a device or an object.

   • Outside distractions such as "rubbernecking"

fig 9:

These actions may suggest that the driver is dozing off or distracted. An immediate warning alert is given to the driver to remind the driver to drive safely. Many road accidents might then be avoided if an alert is sent to a driver who is distracted.

CIRCUIT DIAGRAM

fig 10:

different audio and text indications will be sent. The system has a switch attached to it, and the alert message will only end if the driver shuts off the switch.

A camera monitors the driver's eye blinking, eye closure, facial detection, head position, and other behaviors using a face landmark algorithm and Euclidean distance. The eye closure ratio is calculated by extracting the driver's eye region. If this ratio deviates from the usual ratio, a warning is issued via a buzzer and an LCD display.

The potentiometer acts as a position actuator. In respons to the angle of the steering wheel, the potentiometer changes the resistance in the circuit. When the current passes through the circuit, a voltage drop is created. We can determine the direction and position of the steering wheel by detecting this voltage drop. The change in voltage drop is analyzed to get the steering pattern. The driver receives an immediate warning message to remind him or her to drive carefully

The hardware part of the system consists of:

• Power Supply

• Voltage regulator ICs

• Atmega 328 controller

• Force sensor

• Node Microcontroller Unit

• Wifi module

• LCD Display

• Pressure Sensors

• Potentiometer

• Spst Safety Switch

• Camera

fig 11: LCD Display, Potentiometer, Spst Safety Switch, Atmega 328 controller, Node Microcontroller Unit, Force Sensor

Two voltage regulator ICs are powered by the power supply. The power supply for the microcontrollers and the WiFi module is provided by one of the voltage regulators. The LCD and pressure sensors are powered by the other voltage regulator.

The microcontroller is attached to the pressure sensors, which calculates the fluctuations in the sensors. The pressure fluctuations in the sensors are then monitored. To determine the driver's movement, the sensor with the largest variation is chosen. As a result, the system can tell which portion of the body is moving. The LCD display shows the alert message to warn the driver. The system's alarm will be very specific, ensuring that the driver receives a notification indicating which part needs to be repaired. Depending on the situation,

III.RESULT

In most parts of the world, road traffic injuries are one of the most persistent public health issues, resulting in significant human and economic costs. Around 1.25 million lives are lost each year, and 50 million people are injured in road traffic accidents around the world. It has been proven that dangerous driving conduct is responsible for more than 60% of traffic injuries. This system refers to certain driver's inattention factors while driving to avoid a crash. Our system provides an overview of a system that detects whether a person is drowsy while driving or whether the driver is distracted or not. It also examines the driver's steering style to ensure that he or she is adhering to the speed limit. If that's the case, he'll be notified either by voice or real-time display messages. The overall system effectively supervises it with IoT and computer vision.

OUTPUT Drowsy: if EAR < 0.17
0.24
0.23
0.2
Drowsy (< 0.17)
0.18
0.22
Drowsy (< 0.17)

table 1:Output

fig 12:Hardware Setup

fig 13: eyes landmarks

fig 14:Realtime Database

IV.CONCLUSION

We admit that everybody has a different baseline for eye aspect ratios. In this work, a real-time system that monitors and detects the loss of attention of drivers of vehicles is proposed. The driver's face has been detected by capturing eye landmarks and a warning is given to the driver to avoid real-time crashes. Non-intrusive methods have been preferred over intrusive methods to prevent the driver from being distracted due to the sensors attached to the seat. The proposed approach uses Eye Aspect Ratio and Eye Closure Ratio with adaptive thresholding to detect the drivers drowsiness in real-time. This focuses on the situations when the drivers are used to the strenuous workload and drive continuously for long distances. The proposed system works with the collected data under different conditions. The eye landmarks, the motion of steering patterns, and the seating posture of the driver are captured by the system and are stored in a firebase to provide appropriate alerts. In the future,the data from the real-time database can be used to alert the police, ambulance driver, and hospital authorities and family members if an accident occurs.Thus,the victim can be rescued as soon as possible.These data can also be utilized to learn about each drivers' driving habits and also real time driving updates can be send to a third person (parent,owner of the vehicle.etc..).To alert the driver, a vibration on the steering wheel might be added to the system instead of the buzzer.

V. REFERENCES

[1] R. Patnaik, K. S. Krishna, S. Patnaik, P. Singh, and N. Padhy, "Drowsiness Alert, Alcohol Detect and Collision Control for Vehicle Acceleration," 2020 International Conference on Computer Science, Engineering and Applications (ICCSEA), 2020, pp. 1-5, DOI: 10.1109/ICCSEA49143.2020.9132932.

[2] T. P. Rani, S. K. Sree. Mand P. Sharmila, "Smart Surveillance of Driver Using Machine Learning," 2021 3rd International Conference on Signal Processing and Communication (ICPSC), 2021, pp. 85-88, DOI: 10.1109/ICSPC51351.2021.9451642.

[3] I. Lashkov, A. Kashevnik, N. Shilov, V. Parfenov, and A. Shabaev, "Driver Dangerous State Detection Based on OpenCV & Dlib Libraries Using Mobile Video Processing," 2019 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), 2019, pp. 74-79, DOI: 10.1109/CSE/EUC.2019.00024I.

[4] S. Sathasivam, A. K. Mahamad, S. Saon, A. Sidek, M. M. Som and H.

A. Ameen, "Drowsiness Detection System using Eye Aspect Ratio Technique," 2020 IEEE Student Conference on Research and Development (SCOReD), 2020, pp. 448-452, DOI: 10.1109/SCOReD50371.2020.9251035.

[5] J. Chandiwala and S. Agarwal, "Drivers real-time Drowsiness Detection using Adaptable Eye Aspect Ratio and Smart Alarm System," 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), 2021, pp. 1350-1355, DOI: 10.1109/ICACCS51430.2021.9441756.

[6] S. Cheamanunkul and S. Chawla, "Drowsiness Detection using Facial Emotions and Eye Aspect Ratios," 2020 24th International Computer Science and Engineering Conference (ICSEC), 2020, pp. 1-4, DOI: 10.1109/ICSEC51790.2020.9375240.

[7] Facial Features Monitoring for Real-Time Drowsiness Detection by Manu B.N, 2016 12th International Conference on Innovations in Information Technology (IIT) [Pg. 78-81] https://ieeexplore.ieee.org/document/7880030

[8] Real-Time Drowsiness Detection using Eye Blink Monitoring by Amna Rahman Department of Software Engineering Fatima Jinnah Women University 2015 National Software Engineering Conference (NSEC 2015) https://ieeexplore.ieee.org/document/7396336

[9] Implementation of the Driver Drowsiness Detection System by K. Srijayathi International Journal of Science, Engineering and Technology Research (IJSETR) Volume 2, Issue 9, September 2013

[10] Method for the analysis of posture and interface pressure of the car. By Giuseppe Andreoni, Giorgio C.Santambrogio, Marco Rabuffetti and Antonio PedottiApplied Ergonomics, Vol 33, Issue 6,November 2002, page 511 522

[11] Solmaz, S. & Coskun, Turker. (2013). An automotive vehicle dynamics prototyping platform based on a remote control model car. Turkish Journal of Electrical Engineering and Computer Sciences. 21. 439-451. 10.3906/elk-1109-22.

[12] Rajhans, Neela & Rajhans, R.G. & Ahuja, Dr. (2008). Study of weight distribution on automobile seat through pressure mapping. International Ergonomics Conference, HWWE- 2008,.

[13] S. Gupta and E. Garima, Road accident prevention system using driver's drowsiness detection by combining eye closure and yawning, International Journal of Research, pp. 839842, 2014.

[14] T. Danisman, I. Bilasco, C. Djeraba, and N. Ihaddadene. Drowsy driver detection system using eye blink patterns. In Machine and Web Intelligence (ICMWI), Oct 2010. , 6, 7

[15] O. Khunpisuth, T. Chotchinasri, V. Koschakosai, and N. Hnoohom, Driver drowsiness detection using eye-closeness detection, in 2016 12th International Conference on Signal-Image Technology & Internet- Based Systems (SITIS), pp. 661668, IEEE, 2016.