Anger Mood Detection Based On Face Recognition And Heartbeat

Anger Mood Detection Based On Face Recognition And Heartbeat

Abstract

Anger is one of the feeling that showing strong resentment that can be found in facial expressions, body language, physiological responses and at times in public acts of aggression. So, this project has designed to examine the existence of the individual nature of anger using facial expression and heartbeat. The objective of this project is to warn and help people to control their temper. This can prevent the bad effect towards heart rate, blood pressure, levels of adrenaline and noradrenalin and other physical effects. Matlab- Image-processing is used to analyze facial expressions while heartbeat is measured using a heartbeat detector circuit. The Graphic User Interface (GUI) indicates the angry face result while the Liquid Crystal Display states the faster heartbeat when a person is angry.

1. Introduction

   Anger is one of the sensation that showing strong resentment that can be found in facial expressions, body language, physiological responses and at times in public acts of aggression. An angry person may lose his/her objectivity, empathy, prudence or thoughtfulness and may cause harm to others. When a person is losing his temper, his body reacts just like it does to stress. His heart beats faster to pump increased oxygen, adrenaline and sugar into his bloodstream. His breathing becomes more rapid, his blood pressure rises and his muscles tense. While his face looks beastly and ferocious.

   The main idea of this project is to design a system that can determine the existence of the individual anger that can be operated based on the pulse rate detector and facial recognition, and then displayed the image and pulse using GUI and Liquid Crystal Display (LCD).

   Facial expression communicates information about

   emotions, regulates interpersonal behavior and person perception, indexes physiologic functioning, and is essential to evaluating preverbal infants. [6].

   Images are produced by a variety of physical devices, including still and video cameras, X-ray devices, electron microscopes, radar and ultrasound, and used for a variety purpose including entertainment, medical, business, industrial, military, civil, security and scientific. The goals in each case are for an observer, human or machine to extract useful information about the scene being imaged. An images maybe defined as a two dimensional function, f (x, y) where x and y are spatial coordinates, and the amplitude of f at any pair of coordinates (x, y) is called the intensity or gray level of the image of that point. When x, y and the amplitude value f are all finite, discrete quantity, the image is calling a digital image. Digital image processing allows one to enhance image features of interest while attenuating detail irrelevant to a given application and then extract useful information about the scene from the image [3].

   Matlab can display the values of any matrix as an image. For example, depending on the commands used, Matlab can display the digital elevation values. Displaying matrix as an image in a variety of ways gives further insight into data. By understanding the different image types explained in this article, Matlab can exactly turns matrix values into image pixel colors. Then can control the way Matlab displays the data, and correct images that display incorrectly. In a truecolor image, every image pixel has three values associated with it: the red, green, and blue components. It represents a true color image in Matlab with a three- dimensional array of size M-by-N-by-3. Display functions in Matlab and the Image Processing Toolbox treat such an array as a truecolor image.

   Figure 1. Truecolor image with red, blue, yellow,

   and gray pixels

   Binary images contain only black and white pixels. The output of an image segmentation algorithm is often a binary image, where the white pixels represent the object of interest and the black pixels represent the background. Binary images are sometimes used for text or for line art. They are also used in image processing as a mask. An image mask divides the pixels into two sets. One set, typically the white pixels, contains the locations of interest. For example, the white pixels might indicate detected objects. The other set indicates the background locations. Image Processing Toolbox functions treat a logical matrix as a binary image. A logical matrix contains only 0 (displayed as black) and 1 (displayed as white) [11].

   The expression of the face is a basic mode of nonverbal communication among people. The overall expression of the face is a composite of signs from many sources, such as the bony structure, shapes and positions of features, and color and texture of the skin. The facial muscles produce the varying facial expressions that convey information about emotion, mood, and ideas. Emotion expressions are one primary result of activity by the facial muscles [6].

   The photoresistor or light dependent resistor, LDR, finds many uses as a low cost photo sensitive element. Photoresistors have been made using materials including cadmium sulphide, lead sulphide, and the more commonly used semiconductor materials including germanium, silicon and gallium arsenide. Essentially the LDR or photoresisitor consists of a resistive material sensitive to light that is exposed to light. The photo resistive element comprises section of the material with contacts at either end [4].

2. Methodology

   1. Hardware development

      The heartbeat detector is developed to measure an input data which is the pulse. After the input data was

      detected, it will be sent to the PIC microcontroller to do the comparison. This system basically compared between the input data and the reference data which has been set at 87 bit per minute (bpm) for a normal pulse rate. The heartbeat value then indicated on the Liquid Crystal Display (LCD). The flowchart is shown in Figure 2.

      Start Measuring Input

      Data Microcontroller

      Comparing the No Data

      Yes LCD Display

      End

      Figure 2. Heartbeat detector system

      1. Block diagram. The block diagram is shown in Figure 3. The input stage consists of the pulse sensor circuit that purposely to detect the heartbeat. The value was compared with the reference data in the next stage using the microcontroller before proceeds to the next stage. The references data for normal human heartbeat is 87 bpm. If the detected value is higher than reference data, the person is considered in an angry mood. The output of this system will warn the person that he is in angry state.

         Figure 3. Block diagram for the hardware

         development

      2. Pulse detector circuit. The pulse is taken from the pulse detector circuit as shown in Figure 4. It has two operational amplifiers (Op-Amps) functioned as non-inverting amplifiers that used to increase the input voltage. The input signal is connected to pin 3 of the LM358 and then amplified by both of amplifiers used.

         The high intensity LED used to produce a light towards the LDR. If the intensity of the light is strength, the resistance of LDR will be low while if the light intensity is weak, the resistance of LDR will be high. When a finger is placed between the high intensity LED and LDR, the light intensity will be reduced.

         Figure 4. Pulse detector circuit

   2. Software implementation

      1. Image Data Collection. The image for the database was captured using digital camera in the jpg file type. The images captured from front side of the face in order to give a better performance.

      2. MATLAB image processing. There are several processes to design image detection. First, an initial of face image (the train set) was acquired. The face image was saved as JPEG. Second, the original image was transferred into a multiplication technique. Multiplication is a technique to change the brightness of image. User can play with the number to multiply the original image. After that, the face images was changed into the grayscale to reduce the bit in the image to make the image easier to be analyzed. Then, the input image was converted into threshold values. At the output binary image, there are only two which are

         0 or 1 or represented as black color and white color. This threshold makes the normalized correlation calculation in the template matching easier to be performed and less processing time consumed. Lastly, the value of black pixels by area of image was calculated minus white pixels of image.

      3. Cropping. The image was cropped manually as shown in Figure 5. The size of the eye image is set to 160×160 pixels and these images were saved in folder file of the system for further process.

         Figure 5. The process of cropping the image

      4. Graphic User Interface (GUI). This system was designed using the GUI. It is automatically included templates like push button, static text, edit text, pop-up menu, axes, button group, panel and so on. The desire panel was dragged from the button panel and put into the design space. In the design window, the needed button was selected to be use for the system.

         .

         Figure 6. GUI Command Window

3. Result and analysis

   1. Heartbeat detection

      For the hardware part, when the detection circuit sensed the higher value compared to the reference data, it will trigger a buzzer to give a warning and the LCD will display the pulse reading with the comment indicated. For example if the pulse rate is lower than 87 beat per minute (bpm), the comment displayed as normal and if the pulse rate is higher than 87 bpm, the comment displayed as cool down!!. So the user

      will realize that he is in angry condition and need to relax.

      Figure 7. Heartbeat of normal emotion

      Figure 8. Heartbeat of angry emotion

      Table 1 shows the heartbeat taken whenever a person has different emotions which are angry and no angry. It shows that when a person is angry, the heartbeat goes higher than the reference value.

      Table 1. Heartbeat detection

      Heartbeat (bpm)	Emotion (ANGRY/NORMAL)
      86	NORMAL
      90	ANGRY
      90	ANGRY
      97	ANGRY
      90	ANGRY
      84	NORMAL
      72	NORMAL
      92	ANGRY
      70	NORMAL
      73	NORMAL
      91	ANGRY
      70	NORMAL
      84	NORMAL
      98	ANGRY
      78	NORMAL
      102	ANGRY
      86	NORMAL
      96	ANGRY

   2. Pre-processing analysis

      This part needs to be performed before it is implemented for further processing. The first step after

      loading the image is multiplication. Multiplication is applied to change the brightness of the image. For this task, the image is multiply by a factor of 1.2 to increase the brightness and slightly accentuate the craters. Second step is grayscale conversion. Grayscale conversion is used to reduce the bit in the image that makes the image easier to be analyzed. After that the image is converted into a binary form based on the threshold values. At the output binary image, there are only two which are 0 or 1 and this will make the normalized correlation calculation in the template matching easier to be performed and less processing time consumed.

      Figure 9. From original image to graythresh

      image

      After getting the graythresh image, the pixels of black image by area of image (314 x 235 pixels) minus with the pixels of white image. The white image is like a hole. That is why the black image pixels only needed. Table 2 show that the black color image pixels for smiley face higher than normal face, black color image pixels for happy face higher than smile face and black color image pixels for angry face higher than happy face. Black color represents facial features clearly. That is why for this project, the black color is emphasized more. From this result, it can be concluded that the black color image pixels for angry face are more than

      other faces.

      Table 2. Original image, binary image, value of black pixels and desription

   3. Graphic User Interface (GUI)

      The GUI layout for this project is shown in Figure

      10. The Picture Load button is to load the image. The Measure button is to measure the black pixels image. The Clear button is to clear the image. The Exit button is to exit from the program. The 4 axes is to show the different type of images, the Edit Text box is to show the value of black pixels image and the Formula box is to calculate the value of face image.

      Figure 10. GUI layout

4. Conclusions

   In conclusion, this project comprised two main parts which are software implementation and hardware development. The MATLAB image processing has processed the images and gave the differences of the images. To be specific, it can identify the angry emotion from facial expression. This project also compares the real pulse rate with the reference data. From the experiment result, it is proved that human heart will beat faster than normal when they are angry.

   The advantage of this project is it can give a warning to the user when he become angry either he/she realize about the anger or not. This project also can help the user to control his anger before it become worst, which can give some bad effects on him such as heart disease, high blood pressure, hypertension and stroke. By developing this project, the problem will be overcome and the user does not need to worry about his anger become out of control anymore.

5. Acknowledgment

   This work has been supported financially by the Office for Research, Innovation, Commercialization and Consultancy Management (ORICC), Universiti Tun Hussein Onn Malaysia and some facilities are provided by Faculty of Electrical and Electronic Engineering, Universiti Tun Hussein Onn Malaysia.

6. References

1. Aron Wolfe Siegman, Robert A. Anderson and Tal Berger (1990). The Angry Voice: Its Effects on the Experience of Anger and Cardiovascular Reactivity. American Psychosomatic Society. Psychosomatic Medicine 52:631-643.

2. Gus Q. Zhanga & Weiguo Zhangb,. (2006). Heart rate, lifespan, and mortality risk. The University of Texas Southwestern Medical School, Dallas.

3. Gonzalez, R. C. & Woods, R. E. (2008). Digital Image Processing Third Edition. Upper Saddle Rivr: Pearson Prentice Hall.

4. Ian Poole (2010) Light dependent resistor or photo resistor. Retrieved September 29, 2010, from http://www.radioelectronics.com/info/data/resistor/l dr/light_dependent_resistor.php

5. Jamesick, James R. (2001). Scientific Charge- Couple Devices. SPIE-The international Society For Optical.

6. Joseph C. Hager. (2003). Introduction to the DataFace Site: Facial Expressions, Emotion Expressions, Nonverbal Communication, Physiognomy. Retrieved September 17, 2010, from http://face-and- emotion.com/dataface/general/homepage.jsp

7. Kawamoto, H. (2002). The history of liquid-crystal displays. Proc of the IEEE.460-500

8. Prof. Jeffrey Cohn, Professor Takeo Kanade, James Lien, Yu-Te WuLee Bah Seong (1997). Facial Expression Analysis. University of Pittsburgh.

9. Prof. Lata,Y.V., Tungathurthi,C. K. B., H, Ram Mohan Rao., Dr. Govardhan, A., & Dr. Reddy, C.

   P. (2009). Facial Recognition using Eigenfaces by PCA International Journal of Recent Trends in Engineering, Vol 1, No1.

10. Riaz, Z. & Khanttak, K. (2006). Classification of Human Face Images using Principle Component. In 2nd International Conference on Emerging Technologies Peshawar, Pakistan, 13-14 November 2006.

11. Steve Eddins. (2006). MATLAB Digest-How MATLAB Represents Pixel Colors In 2nd International Conference on Emerging Technologies Peshawar, Pakistan, 13-14 November 2006.