Realtime Drowsiness Detector &Alert System For Sleepy Commuters

Realtime Drowsiness Detector &Alert System For Sleepy Commuters

Aju Thomas1

Dept .of Computer Science and Engineering Mangalam College of Engineering Ettumanoor,India

Harikrishnan B2

Dept .of Computer Science and Engineering Mangalam College of Engineering Ettumanoor,India

Ajay George3

Dept .of Computer Science and Engineering Mangalam College of Engineering Ettumanoor,India

Ms.Sruthy Emmanuel4

Dept .of Computer Science and Engineering Mangalam College of Engineering Ettumanoor,India

Abstract In recent years, driver drowsiness and sleep are a significant causes of road accidents, especially when drivers drive for a long time on highways. Avoiding an accident can be the aim of smart systems nowadays. A robust driver detection system must be designed to alert the driver. These project detections are based on various physical-based techniques that detect features such as eyes state (closed or opened), eye blinking rate, yawning, and head movement. In the proposed project we detect drowsiness using behavioral-based techniques. We have also included face detection and various alert systems more than the existing system.

KeywordsDriver Fatigue,Drowsy Driving,Sleepiness Alertness Monitoring,Vehicle Safety Driver Behaviour

I.INTRODUCTION

Drowsiness refers to the state of feeling a strong urge to sleep and being close to falling asleep, either in a routine pre-sleep state or as a chronic condition where there is no regular rhythm. However, being drowsy while engaging in activities that require constant attention, such as driving, can be dangerous. Driving while feeling sleepy increases the risk of a traffic accident. Therefore, it is crucial to detect and prevent driver drowsiness to reduce road accidents. The primary aim of this project is to create a simulation system that accurately detects whether the driver's eyes and mouth are open or closed. By monitoring eye movements, yawning, and the time the eyes are closed, it is possible to detect early signs of drowsiness and prevent potential accidents. This system could be used in various industries, including transportation and healthcare, to improve safety and diagnose sleep disorders accurately. The study focuses on localizing the eyes and mouth in facial images, which involves analyzing the entire facial image using image processing algorithms to identify eye and mouth locations. The technology then detects drowsiness by determining whether the eyes are open or closed. This project's success could prevent countless

accidents and save lives by detecting drowsiness and enabling timely intervention.

The project aims to develop a drowsiness detection system that can accurately track whether a driver's eyes and mouth are open or closed to prevent accidents caused by driver fatigue. Drowsiness refers to a person's strong desire to sleep, and it can be both a routine state just before falling asleep and a chronic condition where a person remains in that state without a regular rhythm. It can be particularly dangerous when performing activities that require continual attention, such as operating a vehicle, which increases the risk of a traffic collision.Detecting driver drowsiness is a significant challenge in the development of accident avoidance systems, and reducing the effects of drowsiness is crucial for road safety. The project will focus on developing a system that can detect yawning and ocular closure as indicators of drowsiness. The system will use a series of facial picture sequences to determine when a driver is yawning and the proportion of time their eyes are closed within a given period.The localization of the eyes and mouth will also be examined to determine whether they are open or closed using image processing algorithms. This technology has applications beyond the transportation industry, including face recognition, human identification, and security systems.The development of a reliable drowsiness detection system has the potential to save countless lives by preventing accidents caused by drowsy driving and facilitating accurate diagnosis and treatment of sleep disorders in medical facilities.

1. RELATED WORK

   1. Camera-Based Eye Blink Detection Algorithm for Assessing Driver Drowsiness.[1]

      This paper presents a camera-based algorithm for detecting eye blinks while driving, with the goal of assessing drowsiness levels. The algorithm isdesigned to handle the challengesthatarise whendetecting eye blinks in the automotive context, such as distinguishing between eye blinks and glances at the dashboard and accounting for high inter-

      individual differences in palpebral aperture. The algorithm is adaptive, meaning it can adjust to the intra-individual variability of blinks that occurs due to drowsiness. It is based on a threshold for the maximum velocity of the eyelids, which is determined using k-means clustering and updated every five minutes ofdriving. The algorithms accuracy is evaluated through video abelling and the results show that it performs well in awakeanddrowsydriving conditions.

   2. A Cost-Effective Driver Drowsiness Recognition System.[2]

      This Paper discusses developing and evaluating a real-time system for detecting driver drowsiness. Driver drowsiness is a major cause of traffic accidents and can lead to serious injuries and fatalities. Therefore, developing effective systems for detecting and alerting drivers when they are in danger of falling asleep at the wheel is an important area of research.

   3. Eye Gaze Tracking Based Driver Monitoring System.[3]

      This paper discusses the development and evaluation of a system for monitoring a drivers gaze and alerting them if their attention wanders from the road. Maintaining attention while driving is essential for safe operation of a vehicle, as it allows the driver to respond to changing traffic conditions and potential hazards

   4. Head movement-based driver drowsiness detection: A review of state-of-art techniques.[4]

      The paper that examines the current state of the art in the field of head movement-based driver drowsiness detection. The paper is review the various techniques that have been developed for detecting driver drowsiness based on head movements. Driver drowsiness is a major cause of traffic accidents and can lead to serious injuries and fatalities. Therefore, developing effective systems for detecting and alerting drivers when they are in danger of falling asleep at the wheel is an important area of research.

   5. Driver Fatigue Detection Systems.[5]

   This paper examines the current state of the art in the field of driver fatigue detection systems. the paper reviews the various systems and techniques that have been developed for detecting driver fatigue. Driver fatigue is a common cause of traffic accidents and can lead to serious injuries and fatalities. Therefore, developing effective systems for detecting and alerting drivers when they are in danger of falling asleep at the wheel is an important area of research. The paper may examine a range of techniques and systems that use sensors such as cameras, accelerometers, and electroencephalography (EEG) to monitor a driver's behavior and physiological signals and identify patterns that are indicative of fatigue. It may also discuss the performance of these techniques in terms of their accuracy and reliability, as well as any limitations or challenges that have been identified. The goal of the review is likely to be to provide a comprehensive overview of the current state of the art in driver fatigue detection systems and to idetify areas for future research.

2. RESEARCH METHODOLOGY

   In general, the term research methodology denotes a collection of techniques and processes that will be employed to conduct a particular research study.In order to complete this project systematically within the specified time, there are some

   methodologies and activities that need to be planned and followed consistently.

3. PREVIOUS DATA GATHERING AND ANALYSIS

   During this phase of the project, a dataset of drivers was created by capturing images from various angles and in diverse environments (such as day, night, and rain). It was discovered that an effective approach for detecting eye state and yawning is through the use of algorithms. Therefore, several algorithms pertinent to the project were analyzed to aid in its development. The proposed method involves measuring the length of time that a person keeps their eyes closed. If this duration exceeds the typical blink time, it is possible that the person is becoming drowsy. Research has shown that the average blink duration for humans is approximately 202.24ms, while drowsy individuals tend to blink for around 258.57ms.

4. ALGORITHM DESIGN AND DEVELOPMENT VIOLA- JONES FACE DETECTION ALGORITHM

   The Viola-Jones object detection framework is capable of detecting multiple classes of objects, but its primary focus is on identifying faces and facial features. This algorithm employs the concept of rectangle features, where the sum of pixels within rectangular regions is computed. The white rectangles' pixel sum is subtracted from the grey rectangles' sum. A two- rectangle feature, denoted as A and B, is the difference between the sums of two rectangular regions with the same size and shape, horizontally or vertically oriented, and adjacent. A three- rectangle feature, denoted as C, calculates the sum within two outer rectangles and subtracts it from the sum in a center rectangle. Finally, a four-rectangle feature, denoted as D, computes the difference between diagonal pairs of rectangles.

   The PERCLOS algorithm, short for Percentage of Closures, is an algorithm that detects driver drowsiness based on the proportion of time the eyes are closed while driving. The algorithm assumes that when the driver is drowsy, their eyes will be closed for a higher percentage of the time than when they are alert. The algorithm measures the time the driver's eyes are open and closed, calculates a ratio known as the Perclos value, and compares it to a predefined threshold. If the Perclos value exceeds the threshold, it indicates that the driver may be drowsy. The Perclos algorithm is typically used alongside other sensors and systems, such as facial recognition, to provide a comprehensive analysis of the driver's state. It has various applications, including ensuring the safety of drivers and other road users in autonomous vehicles.Feature extraction involves extracting features from preprocessed images using computer vision algorithms such as Histogram of Oriented Gradients (HOG), Local Binary Patterns (LBP), and others.

5. MODEL DEVELOPMENT DEVELOPING THE MODEL USING DEEP LEARNING ALGORITHMS

   Convolutional Neural Networks (CNNs) A Convolutional Neural Network (CNN) is a type of deep learning neural network used in image recognition and processing that is

   specifically designed to process data that has a gridlike topology. CNNs are composed of several layers, including an input layer, a series of convolutional and pooling layers, and an output layer. The convolutional layers consist of a set of filters that scan the input data and extract features from it. The pooling layers combine the features extracted from the convolutional layers, while the output layer produces the desired output. CNNs have been used in a variety of tasks such as image classification, object recognition, and natural language processing. It is used to detect patterns in images, classify objects, and detect drowsiness. In drowsiness detection, CNN is used to analyze images from cameras to detect changes in facial expressions and eye movements that indicate drowsiness. The CNN can also be used to detect subtle changes in the driver's behavior that may indicate drowsiness. For example, if the driver's gaze shifts from the road to the side mirror, the CNN can detect this change and alert the driver. In addition, the CNN can be used to detect the presence of any other objects in the driver's field of vision, such as other cars or pedestrians. By analyzing the driver's face and the environment around them, the CNN can detect drowsiness and alert the driver to take action. Recurrent Neural Networks (RNNs) Recurrent Neural Networks (RNNs) are a type of artificial neural network that is designed to recognize patterns in sequential data. Unlike traditional feed-forward neural networks, RNNs are equipped with a type of memory known as a recurrent state, allowing them to make use of information from previous inputs when processing new ones. This makes them well suited for tasks like natural language processing, speech recognition, and time- series analysis. Recurrent Neural Networks (RNNs) are a type of artificial neural network designed to process sequence data to aid in tasks such as language translation and automatic drowsiness detection. In the context of drowsiness detection, RNNs can be used to analyze video footage or audio signals to detect signs of drowsiness in a driver. The RNNs analyze the data in a temporal sequence, taking into account the past, present, and future frames or signals to identify patterns of drowsiness. In addition, RNNs can also incorporate contextual information such as environmental conditions and the drivers past behaviors to better predict the current state of drowsiness. The RNNs can then output a score that reflects the level of drowsiness and alert the driver if they are at risk of falling asleep at the wheel. 4.6 Model Improvement: Current models only alert the driver with a buzzer alarm.So we are bringing some extra features such as

   • Emergency Message to The Vehicles Owner

   • Nearest Refreshment Area

   • Alerting Trip Members Via Trip Mode

   • Detection Of Accidents Using Accelerometer

   • Providing Emergency SoS Call

   • Face Recognition

   • Automatic Deceleration

6. PROPOSED SYSTEM

   This proposed system for drowsiness detection would utilize a camera that is mounted on the interior of the car to monitor the driver's face. The system would use facial recognition technology to detect signs of drowsiness such as yawning, blinking, or closing of the eyes. When the system detects these signs, it would then trigger an alarm to alert the

   driver and remind them to stay awake. The system would also be able to detect any changes in the driver's behavior and alert them if the driver's steering or speed changes dramatically. The system would also be able to detect any sudden changes in the environment, such as a change in the ambient light or noise level, to ensure the driver stays alert. 3.1 Various Techniques Behavioural or physical based techniques identify the driver drowsiness by detecting features such as eyes state (closed or opened), eye blinking rate,head movement. In most cases, behavioral based techniques follow a particular procedure to implement drowsiness detection. While the percentage of eye closure (PERCLOS) over some time is used to evaluate driver drowsiness by detecting if the driver's eyes opened or closed. And Others proposed a drowsiness detection system by developing a sober drive system using Android smartphones in the real world. The authors indicate that the percentage of eyelid closure (PERCLOS), blink time, blink rate, and eyes states are an effective way to evaluate the driver's drowsiness. Back Propagation Neural Network (BPNN) is used to classify eyes opened or closed. Their test data are taken from five participants, and each one made personal training Neual Network with 60 opened and another 60 closed eyes images. The result shows that the detection accuracy is 95% in good conditions, but the accuracy falls when drivers wearing glasses and under low illumination

   The prototype aims to detect driver fatigue by monitoring physiological signals such as eye blinking. It is crucial for our work to detect eye blinks as they are used to operate the device and trigger events. To achieve this, blink detection is necessary. We can use commercially available blink detectors or incorporate a specific instruction in image processing. This instruction will identify an event called "blink" if no pupil is detected for a predetermined period, which is greater than the average human eye blink duration. A set of operations will follow this event. The proposed prototype has four major modules

   • Acquisition of driver image

   • Pupil detection Tracking of eyes

   • region Analyzing visual behavior

   Detecting driver operation through parameters such as steering wheel movement, accelerator or brake patterns, vehicle speed, lateral acceleration, and lateral displacement is nonintrusive, but not a viable option as it is limited to the type of vehicle and driver conditions. The most practical approach to detect driver drowsiness is by monitoring physiological signals like eye blink. However, using sensing electrodes can be cumbersome for the driver, and continuous perspiration can decrease the device's effectiveness. To overcome this, blink detectors can be integrated with a special instruction in image processing. This instruction will identify an event called "blink" if no pupil is detected for a predetermined period greater than the average human eye blink duration. In this case, the time for the "blink event" must be set at 1 second or above to differentiate it from normal eye blinking.

   Fig 1. System Architecture

   There are multiple ways to detect head movement, and one of them is the angle-based approach, which does not require pattern matching algorithms. In this method, the ACC input signal is initially smoothed to filter out undesired movements. The angle-based model, which we have incorporated into our proposed prototype, is considered effective by researchers. In driver drowsiness systems, head movement is detected using vision-based sensing techniques as in Fig 1. These techniques utilize a camera or other imaging device to identify movements such as nodding or sideways motion of the head. The information obtained is then used to alert the driver to take a break or pull over. In some systems, additional sensors like infrared sensors may be employed to detect head movement. Among the commonly used techniques in driver drowsiness systems is the HeadPose and Motion Analysis (HPMA) method.Steps of detection are shown in Fig 2.This technique involves using a camera to track the driver's head position and motion, and an algorithm to detect any changes that may indicate drowsiness. HPMA can detect head nodding, tilting, and rolling. If any changes in head position or motion are detected, an alarm is triggered to remind the driver to stay alert. The system also triggers an alert if the driver's head is not in the expected position, such as looking straight ahead. This helps detect if the driver is falling asleep or not paying attention to the road, and prompts them to take a break or pull over if necessary. The detection of yawning involves two primary steps. Initially, we identify the yawn component in the face, regardless of the mouth's position. This component primarily corresponds to the opening in the mouth resulting from a wide mouth movement. Subsequently, we employ the mouth's location to validate the identified component.

   Following skin segmentation, we select the largest hole within the face as the potential candidate for a yawning mouth. This hole represents a non-skin area inside the face, which could correspond to the eyes, mouth, or an open mouth. It can be assumed that the open mouth will be the largest of the three during a yawning state. This process allows us to locate a candidate for a yawning mouth.

   Next, we utilize the information extracted from the identified mouth to verify the presence of a yawning mouth. The verification criteria involve examining the number of pixels within the yawning mouth relative to the total number of mouth pixels, as well as considering the relative location of the open mouth in relation to the lips.

   The primary objective of the preliminary analysis is to identify the problem at hand. Initially, the necessity for a new or improved application is determined. Only after this recognition of need can further analysis take place.Once the initial investigation is completed and the requirement for a new or enhanced system is established, various alternative solutions are considered. These potential solutions are referred to as "candidate systems." Each candidate system is thoroughly evaluated, and the most favorable option among them is selected as the solution system, known as the "proposed system."After selecting the proposed system, a feasibility assessment is conducted to determine its practicality and potential benefits for development and implementation. In this context, feasibility refers to evaluating whether it is practical and advantageous to proceed with the system.

   The application has been specifically designed to facilitate easy modifications in the future with minimal effort. It has been observed that the application operates efficiently and effectively. The administrator holds complete control over the application, allowing them to add new signals, locations, and block zones. Additionally, the administrator can also update these components as needed.

   A non-intrusive monitoring scheme has been developed, utilizing a camera to capture optical cues of the driver In Fig 3. This scheme aims to detect drowsiness accurately while ensuring that the driver is not distracted. The core concept of this system involves simulating a drowsiness detection system through image processing techniques, enabling the identification of different levels of drowsiness. Additionally, the system can utilize GPS to notify authorized individuals about the vehicle's location.The implementation of this system can significantly contribute to accident prevention, ultimately saving lives and reducing personal suffering. Drowsiness is determined when the driver's eyes remain closed for more than 80% of a specified time interval. Thus, this project plays a crucial role in detecting driver fatigue in advance and provides alarm signals through sound notifications in our system. To deactivate the warning, a manual deactivation switch is incorporated. Once the system concludes that the driver is falling asleep ,it promptly emits a warning signal.

   Fig 2. Process Of Drowsiness Detection

   Fig 3. Face Detection Using Camera

7. SYSTEM IMPLIMENTATION

   Implementation is the process of bringing and developing operational use and turning it over to the user. Implementation activities extend form planning through from the old system to the new. Implementation of the new system involves three major steps- training of the administrator who may interact with the system, conversion of the old system to the new system and post implementation review.Software has been tested for smoothness and user friendliness. Any person with computer knowledge can operate the system. The only training required is to explain the various features of the system and how to use it. The next step is the implementation conversion. Conversion is the process of performing all the operation that directly results in the turnover of the new system into operation. At first both the new system and existing system were operated in parallel. Use of the new system would not cause disastrous effects.

8. EXPERIMENTAL RESULT

   This chapter provides a detailed explanation of the approach taken to achieve the project objectives and an in-depth overview of the project implementation process. It involves an analysis of each stage encountered in order to successfully complete the project. The chosen methods and their implementation for each stage are elucidated, leading to project success.

   The methodology employed for the driver drowsiness detection project focuses on developing a system capable of monitoring a driver's alertness and detecting signs of drowsiness. This will be achieved through the utilization of machine learning algorithms and computer vision techniques. The system will be trained on a diverse dataset of images captured by cameras installed in the vehicle. These images will undergo processing to extract pertinent features like facial expressions and eye movements. These extracted features will then serve as the basis for training the machine learning algorithms to identify signs of drowsiness. When such signs are detected, the system will promptly alert the driver and suggest appropriate measures to maintain alertness.

   The overarching objective of the project is to create a system that can accurately and promptly detect drowsiness in drivers,

   while providing suitable recommendations to ensure their attentiveness.

9. RESEARCH METHODOLOGY

   Research methodology typically encompasses a series of procedures employed to conduct a specific research study. To ensure the systematic and timely completion of this project, it is essential to establish and adhere to appropriate methodologies and activities.

10. PREVIOUS DATA GATHERING AND ANALYSIS

    During this stage, a dataset of drivers was created by collecting images from various angles and in different environmental conditions (day, night, rain, etc.). It was discovered that employing an algorithm is one of the most effective approaches for detecting eyes and yawning. As part of the project's development, several existing algorithms relevant to this area were reviewed to provide insights and aid in its advancement.

    The proposed method involves measuring the duration of eye closures, and if the duration exceeds the normal blink time, it suggests that the individual may be falling asleep. Through extensive research on human eye blinks, it was determined that the average blink duration for a person is approximately 202.24ms, whereas a drowsy individual tends to have a blink duration of around 258.57ms.

11. FUTURE SCOPE

    The purpose of our model is to detect drowsiness in the eyes of the driver and provide alert signals in the form of audio and messages. It is crucial to respond promptly to these warning signals, especially in low-light conditions, to prevent accidents. To further enhance the system, additional features can be incorporated, such as automatically activating the flashlight in dark environments, thereby improving visibility and safety.

12. CONCLUSION

Driver drowsiness detection is a critical concern that significantly impacts road safety, potentially leading to accidents and loss of life. This conference paper has extensively discussed various methods and techniques utilized for detecting driver drowsiness, encompassing physiological measures, behavioral measures, and machine learning-based approaches. The proposed method outlined in this study has demonstrated its effectiveness in accurately detecting drowsiness in real-time and promptly notifying the driver. Nevertheless, further research is required to enhance the accuracy and reliability of drowsiness detection systems.

In summary, this research underscores the significance of driver drowsiness detection and presents a promising approach to address this issue. The findings of this paper hold potential for the advancement of safer and more efficient driving systems in the future, ultimately bolstering road safety and preserving lives.

REFERENCES

[1] A. Mittal, K. Kumar, S. Dhamija, and M. Kaur, Head movementbased driver drowsiness detection: A review of state-of-art techniques, in 2016 IEEE International Conference on Engineering and Technology (ICETECH), 2016, pp. 903908..

[2] G. Sikander and S. Anwar, Driver fatigue detection systems: A review, IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 6, Art. no. 6, 2018.

[3] A. G. Mavely, J. E. Judith, P. A. Sahal, and S. A. Kuruvilla, Eye gaze tracking based driver monitoring system, in 2017 IEEE International Conference on

Circuits and Systems (ICCS), 2017,pp. 364367.

[4] S. H. Alam, S. S. Tahsin, I. Tarannum, T. Chowdhury, and K.Sarony, A Cost-Effective Driver Drowsiness Recognition System, in 2019 IEEE International Conference on Biomedical Engineering, Computer and Information Technology for Health (BECITHCON), 2019, pp. 2023.

[5] M. H. Baccour, F. Driewer, E. Kasneci, and W. Rosenstiel,Camera-Based Eye Blink Detection Algorithm for Assessing Driver Drowsiness, in 2019 IEEE Intelligent Vehicles Symposium (IV), 2019, pp. 987993