Fatigue Detection System using Image Processing on Video Sequences

Fatigue Detection System using Image Processing on Video Sequences

Neeha Khan

Dept. of Computer Engineering St. Francis Institute Of Engineering

Mumbai, India

Dakshata Panchal

Dept. of Computer Engineering St. Francis Institute Of Engineering

Mumbai, India

Abstract In general, around 20 % of the road accidents are caused due to driver fatigue. A very effective and an intelligent system built into vehicles can minimize the accident death toll caused due to driver fatigue. In this paper, a fatigue detection system, based on image processing in a Real-Time environment, is introduced. The system is used in a running environment of a vehicle, wherein the driver in the vehicle onset is alerted of his drowsiness, thus reducing the rate of accidents. In this model, a camera is fitted in the car, which captures a real time video sequence at 15 fps. The face and the facial features are then detected. The parameters involve the eye state detection and yawning detection along with head rolls. This system is implemented in at morning day light and night time. This system gives high accuracy rate and low error detection with a quick processing of input data. Along with these features it is also cost efficient and easily implementable and installable in every type of vehicle, thus minimizing the number of accidents caused by drivers fatigue to a great extent.

Index Terms Fatigue Detection System, Ada-boost Algorithm, Real-Time environment, canny operator, Harris corner detector.

1. INTRODUCTION

   Driver fatigue is a serious problem resulting in many of the road accidents every year. Driver fatigue results in many death and accidents worldwide. Research shows that driver fatigue contributes approximately up to 20% of the road accidents as reported in [1], and up to one quarter of fatal and serious accidents. These driver driven accidents caused due to the driver fatigue are fatal likely to result in death or severe health injuries and damages as they tend to be high speed impacts as the driver who has fallen asleep cannot reduce the impact due to his reduced vigilance, alertness and concentration. A critical element of safe driving is the drivers reaction time which is reduced due to the drowsiness. As drowsiness reduces this reaction time hindering many other factors impairing the attention based activities. The speed at which information is processed is also reduced by sleepiness. It may be that those who persist in driving underestimate the risk of actually falling asleep while driving.

   The role of these human factors cannot be ruled out. Many recent safety systems are followed to avoid these road accidents. Few passive safety systems which are already incorporated in vehicles which help reduce accidents are seat belts, crashworthy body structures and airbags. To have more stringent safety measures active safety systems are

   required which will alert the driver about his state of drowsiness thus reducing the probability of accidents caused due to driver fatigue.

   In recent years the use of intelligent systems in cars has developed considerably. These intelligent systems use wireless sensor networks to monitor the factors affecting the driver fatigue. Smart cars, which use software techniques to control engine speed, steering, transmission, brake, etc. has improved the quality of driving drastically. Ad hoc networks were the first systems to develop the automatic navigation in cars. A noticeable weakness of these systems is that their responses to environmental changes are not applicable in the real running time. It is especially important in driving where time is a critical factor. Another method to check the driver fatigue is monitoring the physical condition and facial expressions of the drivers, which wireless sensor networks are unable to process and transmit these information with adequate precision.

2. RELATED WORK

   Up to now, leading car companies have proposed various systems to prevent drowsiness of drivers at the time of driving especially at night, all of which have their merits and weaknesses. These techniques can be classified into three

   main categories:

   • Models based on mathematic and computation.

   • Monitoring systems of the driver and the car based on sensor networks.

   • Systems based on detection of the driver fatigue

     Liu and Hosking (2009) proposed a system to detect driver's alertness [2]. In terms of evaluation and prediction of driver's sleepiness and alertness, Barbato et al [3] attempted to analyze the alpha signals of sleep based on EEG graphs. Connor et al (2002) [4] explored the risks associated with the drowsiness of the drivers of heavy vehicles and proposed some strategies based on intelligent systems.The real time monitoring system for predicting driver fatigue was proposed by Ji et al. [6] Bergasa (2006)

     [7] developed a real time monitoring system to detect driver's alertness. Johns (2003) proposed a new method to monitor driver's drowsiness and fatigue based on blink rate [8]. Four years later, he tested the instant drowsiness of the driver using infra-red and analysis of the reflected rays from the eyes [9][10]. Fuzzy fusion was a technique proposed by Damousis et al. to show the likelihood of accidents by

     measuring the activities of eyelids. Based on the changes created in facial expressions of the driver when he/she is about to fall asleep while driving, the smart system is designed with high processing accuracy consisting of three algorithms for detection of driver drowsiness

3. THE PROPOSED SYSTEM

   The proposed algorithm conducts the detection process by recording the video sequence of the driver's and image processing techniques using new proposed algorithms to give a better reliability and efficiency.

   Fig.1 Proposed driver Fatigue Detection System

   In this system the face detection along with the feature extraction is done with the help of Viola and Jones algorithm. The eye lid detection along with the yawn detection is done with the help of Canny Operator along with the Harris Operator. Based on the training which will be given to the system the threshold is decided which will in turn detect the fatigue thus alerting the driver with an alarm. This proposed system will be improvement.

4. SEGMENTATION OF FACE AREA

   The extraction of face images from the video and the facial features especially of the eye region and mouth region are done via the efficient face detection method using AdaBoost based real time face detector an approach of viola and jones alogorithm. This method uses Haar-like features. This helps to classify the image into two regions that is a face region and a non-face region. The Viola-Jones face detector contains three main ideas that make it possible to build a successful face detector that is able to run in real time: the Haar-like features, classifier learning with AdaBoost, and the attentional cascade structure.

   • Haar-like features: Offers a new display of image called Integral Image which enables Haar-like features to be computed rapidly.

   • AdaBoost Learning Algorithm: Offers a learning algorithm based on AdaBoost, which is able to select few Haar-like features from a larger set and will be transformed into a suitable classifier.

   • Cascade Classifier: Offers a method for combining simple to complex AdaBoost classifiers increasingly, which enables background regions of image to be discarded by primary classifiers while more complicated AdaBoost classifiers are required for the face-like regions[11].

   This approach is able to detect faces with high speed and the implementation is easier.

   Fig.2 Final structure of developed face detection system [11]

   The sub-windows are first passed through the first layer or stage in which all sub-windows will be classified as faces or non faces. The negative results will be discarded while remaining positive sub-windows will trigger the evaluation of the next stage classifier [11]. The sub-widows that reach and pass the last stage are classified as face. Each stage consists of several techniques. The last stage is the validation of the detected faces which is done via the validation process which uses the morphological operators.

   Fig.3 Extracted face and facial features using Viola and Jones.

5. EXPERIMENTAL RESULTS

   1. EyeDetection

      The eye state is detected using a combination of canny operator and Harris Corner Detector. These two being the best operators are used for faster and accurate results. Thus determining the open state and closed state of the eye which will help to know the drowsiness level of the driver. The threshold is applied once the eye is determined as closed for a sequence of images. This is a novel approach which is faster and more accurate with lesser error detection rate.

      Fig.4 Experimental result showing detection of drowsiness when eyes are closed.

   2. Yawn Detection

   Yawning is another factor which counts to the drowsiness level. More the number of yawns more precautions should be taken by the driver in order for a safe driving and the driver should be alerted with the help of the alarm. Another sign of fatigue during driving, which is manifested in a person's face, is frequent yawning that is due to body reflexes when a person is exhausted and about to fall asleep. Various systems have been proposed for measuring yawning some of which are slow and time consuming while

   Fig.5 Experimental result showing detection of drowsiness when yawning.

   others are not very accurate in separating the mouth area at the time of yawning. Thus, an efficient technique is needed that is able to display the changes in face configuration and detect the yawning.

   In this system a simple and a very efficient method is used. The method being similar to that of the eye state detection wherein it uses a combination of the canny operator and the Harris Corner Detector. Thus giving the results at a faster rate reducing the errors and alerting the driver of his or her drowsiness.

6. CONCLUSION

   The system proposed in this paper is a hybrid of three different algorithms which are the Viola and Jones algorithm, algorithm of canny operator and Harris corner detector. These algorithms have an acceptable level of performance and an improved average accuracy of 97%. As human errors due to driver fatigue has a high fatality rate for road accidents this system justifies as it uses alarms to alert the driver onset about his state of drowsiness. High detection rate and a much lesser error detection rate distinguish this mentioned system from the existing ones. More and Further developments in this field can save millions of life across the globe.

7. FUTURE WORK

   In addition to the work mentioned in this paper, along with the driver being alerted about his fatigue a system can be developed wherein the vehicle can itself detect the running traffic environment. Based on the detection of the running traffic the vehicle can decide to park the car to the emergency lane avoiding collision and accidents with the cars on the run.

   Hence the future applications can include an automated driving system that can park the car in the emergency lane in case of the driver fatigue level crossing an acceptable threshold.

   Fatigue Detection can find many more application in various fields such as commercial airlines, air force and also in racing. Most importantly in commercial airlines where there are more lives at stake.

8. REFERENCES

1. Wen-Bing Horng* and Chih-Yuan Chen A Real-Time Driver Fatigue Detection System Based on Eye Tracking and Dynamic Template Matching, Tamkang Journal of Science and Engineering, 2008.

2. C. C. Liu, S. G. Hosking, "Predicting driver drowsiness using vehicle measures, recent insigllts and future challenges," J. safety Res, 40 (4 ) , pp 239 -245.2009

3. G. Barbato, G. Ficca, M Beatrice, M Casiello, G. Muscettola and F.Rinaldi, " Etfects of sleep deprivation on spontaneous eye blink rate and alpha EEG power," BioI Psychiatry. 1 995; 38(5):pp 340 – 341 .1 995.

4. J. Conner, R Norton and et aI., "Driver sleepiness and risk of serious injury to car occupants: population based case control study," Br. Med.J., 324 1 1 2 5, 2002.

5. D. Dinges, G. Maislin, "Pilot test fatigue management t echnologies," (No. Issue 1 992 ): US T ransportation Research Record. pp 66 -74, 2005.

6. Q. Ji, Z. Zhu, P. Lan, "Real-time non int rusive monit oring and prediction of driver fatigue". IEEE Trans Veh T echnoI53(4 ): 1052- 1068. 2004.

7. L. M. Bergasa, J. Nuevo, "Real-Tinoe system for monitoring drivervigilance," IEEE Transactions on Intelligent Transportation Systems, 7(1 ) , pp 63-77. 2006.

8. M. W. Johns "The Amplitude velocity Ratio of Blinks: A new method for monitoring drow;iness," Paper presented at the 7fu A n nual Conference, Associated Professional Sleep Societies (APSS) . 2003.

9. M. W. Johns, A T lker, "Monitoring eye and eyelid movement by infrared retlectance oculography to measure drowsiness in driver, "Somnologie, 1 1 , pp 234 – 242. 2007.

10. 1. Damousis, D. Tzovaras, "Fuzzy fusion of eyelid activity indicators for hypovigilance-related accident prediction". IEEE Trans Intell Transp Syst 9(3) :491 . 2008

11. An Efficient Face Detection Method Using Adaboost and Facial Parts Yasaman Heydarzadeh, Abolfazl Toroghi Haghighat Computer, IT and Electronic department Azad University of Qazvin Tehran, Iran heydarzadeh@ qiau.ac.ir , haghighat@qiau.ac.ir