Firefighting Robot with Face Detection System

Firefighting Robot with Face Detection System

Saritha C

Department of Electrical Engineering, University Visvesvaraya College of Engineering, Bangalore University, Bangalore, India

B. P. Harish

Department of Electrical Engineering, University Visvesvaraya College of Engineering, Bangalore University, Bangalore, India

Abstract Multi-sensor fire detection and a face detection system are developed for a fire fighting robot proposed in this work, to rescue human beings caught in fire. Various sensors like fire, smoke and infrared sensors have been incorporated in this robot. The infrared sensor detects any obstacle present in the path of the robot to help the robot navigate.While the fire sensor detects the fire in places like home, factories etc., the smoke sensor detects the smoke. The smoke can be caused either due to the fire or due to the leakage of flammable gases or flammable liquids. If the fire is detected, a water spraying mechanism is triggered. The robot is designed with a water tank and an attached pump. When the fire is detected, the relay gets energized and supplies power to the pump. The pump starts spraying water thereby extinguishing the fire. The face detection system alerts the presence of human beings caught in fire to facilitate their rescue.Text messages are displayed on the LCD upon all events to alert the operator.

Keywords: Firefighting, Water spraying mechanism, Face detection system.

1. INTRODUCTION

   Fire is a critical part of life on earth and a vital part of many ecosystems. But fire has the potential to become a destructing agent when uncontrolled. Uncontrolled fires are very dangerous because they spread rapidly, ruin property and bring suffering and loss to human life. Hence, it is necessary to deploy technology for early detection of fire for effective and ease of control. A fire accident can be natural or man- made. It is very essential to detect the fire at the earliest so that a fire disaster can be avoided as it provides precious time to execute emergency measures to control the fire and limit the damage. Usually fire alarm systems are the widely employed techniques to alert the occurrence of fire. But these systems have certain limitations. There are times when they become too sensitive resulting in false alarms. These false alarms result in panic and wastage of huge amount of human resource and energy. In order to address all these issues efficiently, there is a need for an autonomous firefighting robot which gives true fire alerts and initiates firefighting measures to minimize the loss from the fire accidents.

   An overview of the prior work related to firefighting robot is discussed. A mathematical model based on Linear Quadratic Regulator-Genetic Algorithm (LQR-GA)is proposed to explore the characteristics of a two-wheel self-balancing robot and control its behaviour on a planar surface and on aninclined plane and to impart self-balancing capability [1]. A methodology for firefighting robots to find the shortest and optimized path for searching the target within the workspace is proposed using Particle Swarm Optimization (PSO)

   algorithm. Smoke sensors detect the fire position and perform the fire suppressing action by eliminating all the flammable sources in the fire environment to prevent further damages. Though the system is fast acting, its maintenance is complex and expensive [2]. An automatic fire detection system based on multi-sensor fire detection using adaptive fusion algorithm is presented to compute weights for an artificial neural network based control [3]. A robot using an obstacle avoidance sensor and flame sensor is designed for fire detection. The robot moves along a known path and upon detecting the fire, approaches the fire and extinguishes it [4]. A fire detection method based on video streaming, referred to as Moving Object Detection Video based Fire Detection (VFD) algorithm, is proposed by considering the smoke and the flames as the moving object. The low-cost system is inefficient in finding the fire in both indoor and outdoor environments as no sensors are used [5]. A neural network based multi-sensor fire detection algorithm is developed for ship safety using temperature and smoke sensor signals [6]. A fire detection system based on colour video input using the spatial, temporal and spectral properties of fire is proposed for its detectionin real-time [6]. A genetic algorithm based automatic fire detection system with a stochastic signal model for fire signal is presented [7]. A system with smoke, flame and temperature sensors is described to detect fire and diagnose the sensor failure using adaptive sensory fusion method [9], [10].

   It is evident, from the above literature that all the robotic systems developed for fire detection and control fail to support the rescue of human beings caught in fire. The proposed work overcomes this limitation by not only detecting and extinguishing the fire, but also includes a face detection system to detect the presence of human being caught in fire.

   The remainder of the paper is organized as follows: Section II presents the architecture and process flow of firefighting robot. Section III explains the face detection algorithm. While Section IV presents the results obtained, Section V concludes with an outline of future work.

2. ARCHITECTURE AND PROCESS FLOW OF

   No

   No

   Fire Sensor

   Fire Sensor

   Pump

   Pump

   Vehicle movement signal active

   Vehicle movement signal active

   Smoke Sensor

   Smoke Sensor

   FIREFIGHTING ROBOT

   Start

   Buzzer

   Arduino controller

   IR Sensor

   Activate all components

   Buzzer

   Arduino controller

   IR Sensor

   Activate all components

   UART

   Face Detection System

   UART

   Face Detection System

   Relay

   Relay

   LCD

   LCD

   Read sensor

   Read sensor

   Fig.1: Block diagram of firefighting robot

   The block diagram of the proposed fire fighting robot is as shown in Figure 1. This includes Arduino Uno controller, a set of sensors, LCD display, a buzzer, a motor drive, a relay and a Universal Asynchronous Receiver/Transmitter (UART). A four-wheeled robot is designed for firefighting application in order to provide for its mobility and speed. Arduino Uno reads the digital and analog values from various sensors and controls the movement of the robot. The infrared sensor sends infrared light which is reflected back by any object in its range and its output goes high indicating detection of an object. Upon object detection, the Infrared sensor gives an output to the Arduino Uno controller to stop the robot. The fire sensor used to detect the fire is connected to the controller through the GPIO port. The fire sensor detects fire over a range of detection angle of 60°. The MQ-7 smoke sensor, coated with a sensitive material called Tin dioxide (SnO2), has lower conductivity in clean air. The conductivity of the sensor increases with the increase in the concentration of the Carbon monoxide (CO) gas in the air. When the gas is detected, the conductivity increases, therefore the voltage also increases, detecting the carbon monoxide gas. A buzzer is included to generate loud sound alerts and a LCD is used to display the text alerts. The water tank contains a pump connected to the COM port of the relay which is triggered by the input voltage of 12 V DC.

   The flowchart of the firefighting robot is as shown in the Fig.

   1. Upon power up, the relay gets energized, the motors attached to the two wheels of the robot start running, setting the rbot in motion. If any obstacle is detected, the robot immediately stops, waits for programmed amount of delay and the controller sends a signal to display a text message. When the fire is detected, the controller receives the signal from the sensor, water is pumped to extinguish fire and the

      Yes

      Stop for some time

      Robot will stop

      Robot will stop

      Sprinkle the water and display in LCD

      Sprinkle the water and display in LCD

      No

      Run face detection code

      Run face detection code

      No

      Any obstacle detected?

      No

      Is fire or smoke detected?

      Yes

      Is face detected

      ?

      Send information to controller and display in LCD

      Send information to controller and display in LCD

      Yes

      Stop

      buzzer generates loud sound alert. If smoke is detected, a text message is displayed. If the fire or smoke is detected, face detection mechanism is triggered to extract the image by capturing the video from the webcam. Snapshots are taken from the video and are used to detect the presence of human face. Once the face is detected, a text message is displayed.

      Fig.2: Flowchart of firefighting robot.

3. FACE DETECTION SYSTEM

   The Viola-Jones algorithm is proposed as the first real time object detection framework to detect the faces efficiently, if the face is pointing at the camera that are frontal upright. The Viola-Jones algorithm [11] has 4 stages: Haar feature selection, creation of integral image, Adaboost training, and cascading classifier. There are 5 rectangular Haar like digital image features of Type 1 to Type 5, having more than one rectangular area, as shown in Fig. 3. All human faces have some common properties as follows:

   • The eyes are darker than the upper cheeks.

   • The nose region is brighter than eyes.

   • The lips are darker than its upper and lower region. These common regularities are matched with the human face. The integral image is created by adding all the pixels to left and above as shown in Fig.4. The addition of pixels is carried using the pixels which overlap with corners values in the input image.

   Fig.3: Different types of Haar features

   Fig.4: The integral image

   Viola -Jones considered a standard 24×24 pixel window with a total of 162,336 features, out of which, not all of them are relevant. All the features are not expected to give high values when scanned across the input image. In order to extract only the best features, Adaboost algorithm is used to combine all weak classifiers to produce a strong classifier. A weak classifier is simply a feature whose probability to find in a face is low.The Adaboost algorithm determines the best feature based on best value of threshold and the polarity. In order to achieve this, every new weak classifier needs to evaluate all features on every training example for finding the feature that performs the best. The Adaboost training is carried out by giving examples of positive and negative faces. It rejects non-faces instead of detecting the faces as discarding a non-face is faster than finding a face. The cascaded classifier consists of many stages with strong classifier present in each stage, which determines whether a given sub-window is a face or non-face. If the sub-window is passed on to more stages of the classifier, then the probability that it is a face is high. Fig. 5 shows the cascaded classifier with multiple stages. The MATLAB code implementing

   Viola-Jones algorithm of face detection system runs all the time, looking for human beings caught in fire to rescue them.

   Stage 1 Is input a face?
   		Maybe
   Definitely not!

   Stage 1 Is input a face?
   		Maybe
   Definitely not!

   Stage 2 Is input a face?		Maybe
   Definitely not!

   Stage 2 Is input a face?		Maybe
   Definitely not!

   Input

   Discard input Discard input

   Fig.5: Cascaded classifier with multiple stages

4. RESULTS AND DISCUSSION

   Fig.6: A prototype of firefighting robot

   The firefighting robot is designed and its prototype is developed as shown in the Fig 6. The Viola-Jones algorithm is successfully implemented to detect the human face. The various functionalities of the firefighting robot are successfully demonstrated. The developed indoor firefighting robot can detect fire and extinguish it with an attached water pump using the sensors incorporated. The Arduino code and the motor controller run simultaneously to regulate the movement of robot. The output of the face detection system, when face is detected and when face is not detected, is shown in Fig. 7 and Fig. 8, respectively.

   Fig.7 Output of Face Detection System, when face is detected.

   Fig.8 Output of Face Detection System, when face is not detected.

5. CONCLUSIONS AND FUTURE WORK

Multi-sensor fire detection and a face detection system are developed for the proposed fire fighting robot, to rescue human beings caught in the fire. Various sensors like fire, smoke and Infrared sensors have been incorporated in this robot. If the fire is detected, a water spraying mechanism is triggered to extinguish the fire. Text and sound alerts are issued upon all events to alert the operator.

Future work can include the transformation of the experimental robot prototype into a practical robot, which requires improvement in its overall performance, specifically in payload and controllability. Various control algorithms like machine learning and neural network can be used to improve the efficiency and stability.

REFERENCES

1. Pannaga R.M. and B.P. Harish, Modelling and implementation of two- wheel self-balance robot, International Journal of Electrical, Electronics and Computer Science Engineering, Vol. 4, Issue 6, pp. 33- 40, December 2017.

2. D. Mihiraj Ranaweera, K.T.M. Udayanga Hemapala and A.G.Buddhika

   P. Jayasekara, A Shortest Path Planning Algorithm for PSO Base

   Firefighting Robots, 4th International Conference on Advances in Electrical, Electronics, Information, Communication and Bio- Informatics (AEEICB-18), 2018, pp. 1-5.

3. Kuo. L. Su, Automatic Fire Detection System Using Adaptive Fusion Algorithm for Fire Fighting Robot, IEEEInternational Conference on Systems, Man, and Cybernetics October 2006, Taipei, Taiwan, pp. 966- 971.

4. N. Khoon, P. Sebastian. Autonomous Fire Fighting Mobile Platform, International Symposium on Robotics and Intelligent Sensors, 2012, pp. 1145-1153.

5. A.E. Çetin, K. Dimitropoulos, Video fire detection Review, Digital Signal Processing, December 2013, pp. 1827-1843.

6. W. Xihuai, X. Jianmei and B. Minzhong, "A ship fire alarm system based on fuzzy neural network,"in Proceedings of the 3rd World Congress on Intelligent Control and Automation, Vol. 3,2000, pp. 1734

   -1736

7. Healey, G., Slater, D., Lin, T., Drda, B. Goedeke and A. D., "A system for real-time fire detection," in Proceedings ofIEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1993, pp. 605-606

8. Neubauer A., "Genetic algorithms in automatic fire detection technology," Second International Cnference On Genetic Algorithms in Engineering Systems: Innovations and Applications, 1997, pp. 180- 185.

9. R. C. Luo, K. L. Su and K. H. Tsai, "Fire detection and Isolation for Intelligent Building System Using Adaptive Sensory Fusion Method," Proceedings of The IEEE International Conference on Robotics and Automation, 2002, pp.1777-1781

10. R. C. Luo, K. L. Su and K. H. Tsai, "Intelligent Security Robot Fire Detection System Using Adaptive Sensory Fusion Method," The IEEE International Conference on Industrial Electronics, 2002 (IECON), pp.2663-2668.

11. Paul Viola, Michael Jones, Rapid Object Detection using a Boosted Cascade of Simple Features, Conference on Computer Vision and Pattern Recognition, 2001, pp. 511-518.