FACS Based Eye Movement Recognition using Eigen value Decomposition

FACS Based Eye Movement Recognition using Eigen value Decomposition

Vaishali R. Lokhande

Department of Computer Science and Information Technology

Dr. Babasaheb Ambedkar Marthwada University Aurangabad, India

Bharti W. Gawali

Department of Computer Science and Information Technology

Dr. Babasaheb Ambedkar Marthwada University Aurangabad, India

Abstract – This paper attempts to present an Eigen vector based system to recognize the upper facial expressions. In the approach, the images were accomplished and properly resized. The computation of Eigen vector followed by Eigen value, for recognition of facial expressions is done. The Euclidean Distance has been calculated for getting accuracy in the recognition.

Keywords – Eigen value, Eigen vector, Euclidean Distance and Facial Action Coding System (FACS).

1. INTRODUCTION

   Human face carries wider powerful information while communicating with one another. Facial expression provides sensitive cues about emotion [1], regulates signals to speech production and reveals brain function [2]. Thus has ability to recognize and can capture facial expression.

   The [3] identifies that 7% of human communication information is done by verbal, 38% by non-verbal elements of speech and 55% by facial expression. To classify facial expressions into different categories, it is important to extract facial features which assist in identifying exact expressions [4]. The Facial Action Coding System (FACS) is an efficient, objective method to describe facial expressions [5]

   The Charles Darwin, suggest expression of emotions as universal [6]. Later on Ekman and Friesen describes the emotions can be categorized into six, happiness, sadness, anger, surprise, disgust and fear, also known as the six basic expressions [7, 8].

   In this paper, we have focused on computational model of facial expression recognition system based Eigenvector and Euclidean distance. The proposed approaches, considers a set of 10 images for happiness, anger, surprise, disgust and neutral, is processed and computed Eigenvectors are stored as in training phase. Similarly, a set of 10 images for testing phase is stored with calculated Euclidean distance of the Eigen vector. The classification of expression is considered with the help of minimum Euclidean distance.

2. LITERATURE REVIEW

   The recognition of facial expression is one of the challenging tasks but can be recognized by using one, or combination of techniques. Most of the researchers, apply Eigen faces approach due to its simplicity, speed and learning capability, that seems to classify the different features of

   images by considering a set of eigenvectors which has been derived from the covariance matrix of the faces with the help of principal component analysis (PCA) [8], [9], [10] and Artificial Neural Network (ANN) [11], [12]. Further these values are computed for obtaining minimum Euclidean distance and found to be efficient to produce accuracy of 95% to recognize emotions [6], [13].

   Reference [5] concentrates on the eigenvectors of covariance matrix with the help of row and column mean of face image in RGB plane and set of feature vector were generated to apply PCA technique and achieved 99.60% on the ORL database and 80.60% on their own created database of recognition. Similarly, [14] considers the Eigen value decomposition technique for describing different Action Units (AUs) of the faces and obtained 90% accuracy. The facial recognition rate can be extended by considering the effectiveness of Eigen space method as it is the modified form of Eigen face method, and is described by [8].

   A separate Hidden Markov Model (HMM) can useful for recognizing facial expression and related action units (AU) and also can be combined with Artificial Neural Network (ANN) to obtain AU combination [15], [16], [17]. The Support Vector Machine (SVM) replaces with hybrid SVM and combined with Hidden Markov Model (HMM) to achieve the extended output precision of AUs by 4.5% and 7.0% [18]. Similarly [19], [20], [21], [22] obtained the accuracy rate around 86% for AU and 89% for AU combination.

   1. Eigen values and Eigen vectors

   We focused on the use of Eigen values and Eigen vectors for expression recognition with Euclidean distance. Eigen values are a special set of scalars associated with a linear system of equations, may be sometimes called as characteristic roots, characteristic values [23], proper values or latent roots [24]. Each Eigen value is paired with a corresponding so called eigenvector [25].

   The steps for computation are as follows:

   1. Prepare an initial set of face images [X1,X2,,Xn]

   2. Compute average of the given set:

      X=(X1+X2+.+Xn)/n

   3. The average face is subtracted from each face:

      Xi = Xi X where i=1, 2,,n

   4. Calculating the covariance matrix using the formula:

      Cov = X . XT

   5. Calculating the Eigenvectors and Eigen values of the covariance matrix:

3. PROPOSED METHOD

   1. Image Procurement:

   Some of the images are procured from the well known Paul Ekmans database, which portray different facial expressions. These images are properly resized and stored in the training as well as testing database set. For the further processing these images are converted into its equivalent Gray scale and stored in matrix form.

   The Eigenvector based Expression Recognition System is 2. Cropping:

   shown in Figure1:

   Training Image Set

   Testing Image

   Image Acquisition

   Crop region of interest from the whole face

   Resized the images

   Calculate Mean

   Calculate Covariance

   Calculate Eigenvector

   Store obtained feature in vector set

   Calculate Euclidean Distance

   Expression Recognition

   Fig 1: Eigenvector based Expression Recognition System

   We have focused on upper facial recognition, thus directed towards combined left and right eyes. This region of interest has been cropped with proper resizing.

   1. Use of Statistical technique:

      Calculate the mean, covariance and eigenvectors of the image as given in steps of computation above. Five significant Eigenvectors are chosen from both testing and training images which are sorted in the descending order of the associated Eigen values and are stored in separately in the array.

   2. Classification:

   The classification of facial expression is based on Euclidean distance (ED) which is obtained by using the formula [6], Euclidean Distance (ED) = (x2 x1)2

   Here, the eigenvectors of testing image is subtracted from the eigenvectors of training image. Each expressions minimum Euclidean distance is calculated from the set of observations and is expressed in tabular form as shown in Table I.

   TABLE I. COMPUTED EUCLIDEAN DISTANCE FOR RECOGNIZING EXPRESSION

   Test image	Training Image	Euclidean Distance
   		ED1	ED2	ED3	ED4	ED5
   Combined Left and Right Eye		0.2559	0.0065	0.0296	0.0354	0.0305
   		0.2541	0.2624	0.2930	0.1240	0.1157
   		0.2745	0.2642	0.2914	0.2540	0.2543
   		0.2537	0.2515	0.2828	0.1532	0.1064
   		0.2478	0.2570	0.2882	0.2966	0.1344
   Result obtained from minimum ED		Disgust	Anger	Anger	Anger	Anger

   Observing the minimum value of Euclidean distance among all columns, the expression that occurs maximum times are considered as a facial expression of the testing image.

   In above table 1, based on the minimum Euclidean Distance, Anger expression occurs four times as compare with Disgust. Thus we can conclude that the expression of test image is having Anger expression.

4. CONCLUSION

Eye carry major role for expression recognition. The testing process of each expression with both the eyes describes that the system is effective for recognizing the expression. The result found to be efficient and is 70%. The accuracy of recognition rate can be enhanced if full face is considered.

REFERENCES

1. James J. Lien, Takeo Kanade, Jeffrey F. Cohen, Ching-Chung Li, Automated Facial Expression Recognition Based on FACS Action Units,Copyright 1998 IEEE, Published in the Proceedings of FG98, April 14-16, 1998 in Nara, Japan.

2. James J. Lien, Takeo Kanade, Jeffrey F. Cohen, C. C Li, A Multi-Method Approach for Discriminating Between Similar Facial Expressions, Including Expression Intensity Estimation, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 1998.

3. Mehrabian. A, 1968, Communication without Words, Psychology Today, Vol. 2, No.4, pp 53-56.

4. Rehmat Khan, Rohit Raja, Neural Network associated with recognition of Facial Expressions of Basic Emotions, International Journal of Computer Trends and Technology- Volume 3 Issue 4 2012.

5. Saumil Srivastava, Real Time Facial Expression Recognition Using A Novel Method, The International Journal of Multimedia & Its Applications (IJMA) Vol.4, No.2, April 2012.

6. Jeemoni Kalita and Karen Das, Recognition of Facial Expression Using Eigenvector Based Distributed Features and Euclidean Distance Based Decision Making Technique, (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 4, No. 2, 2013

7. P.Ekman, E. R. Sorenson, and W. V. Friesen, Pan cultural elements in Facial displays of emotion Science, New Series, Vol. 164, No. 3875, pp . 86-88, April 4, 1969.

8. G. R. S. Murthy, R.S.Jadon, Effectiveness of Eigenspaces for Facial Expressions Recognition, International Journal of Computer Theory and Engineering, Vol. 1, No. 5, pp. 638- 642, December, 2009, 1793-8201.

9. Maja Panticand Leon J. M. Rothkrantz, Facial Action Recognition for Facial ExpressionAnalysis From Static Face Images, IEEE Transactions on Systems, Man and Cybernetics – Part-B : Cybernetics, Vol-34, No.3, June 2004-1449.

10. Mamta Dhanda, Face Recognition Using Eigenvectors from Principal Component Analysis, International Journal of Advanced Engineering Research and Studies E-ISSN2249 8974, IJAERS/Vol. I/ Issue II/January-March, 2012/37-39

11. Y. Tian, T. Kanade, J. Cohn, Recognizing action units for facial expression analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 23 (2) (2001) 97115.

12. Ann M. Kring, Denise M. Sloan,The Facial Expression Coding System (FACES): Development, Validation,and Utility, Psychological Assessment Copyright 2007 by the American Psychological Association2007, Vol. 19, No. 2, 210224.

13. Maja Panticand Ioannis Patras, Dynamics of Facial Expression: Recognition ofFacial Actions and Their Temporal Segments from Face Profile Image Sequences, IEEE Transactions on Systems, Man and Cybernetics – Part-B : Cybernetics, Vol-36,

    No.2, April 2006

14. Kohei Arai,Method for Face Identification with Facial Action Coding System: FACS Based on Eigen Value Decomposition, (IJARAI) International Journal of Advanced Research in Artificial Intelligence, Vol 1,No. 9,2012.

15. I. Cohen, N. Sebe, A. Garg, L. Chen, T.S. Huang, Facial expression recognition from video sequences: temporal and

    static modeling, Computer Vision and Image Understanding 91 (2003) 160187.

16. M. Yeasin, B. Bullot, R. Sharma, From facial expression to level of interests: a spatio-temporal approach, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2004.

17. Mahmoud Khademi, Mohammad Taghi Manzuri-Shalmani, Mohammad Hadi Kiapour, and Ali Akbar Kiaei, Recognizing Combinations of Facial Action Units with Different Intensity Using a Mixture of Hidden Markov Models and Neutral Network.

18. M.F. Valstar and M. Pantic, Combined Support Vector Machines and Hidden Markov Models for Modeling Facial Action Temporal Dynamics

19. Marian Stewart Bartlett, Gianluca Donato Javier R. Movellan,Face image analysis for expression measurement and detection of deceit, 1999 6th Joint Symposium on Neural Computation Proceedings

20. Jake Grosek and Grant Roush, Eye Extraction and Detection, Computational Methods for data analysis, Final project, March- 2010

21. Tim Chuk, Alvin C. W.Ng, Emanuele Coviello, Antoni B. Chan, Janet H. Hsiao, Understanding eye movements in face recognition with hidden Markov model.

22. Jeffrey F. Cohn, Tomas Simon Kruez, Iain Matthews, Ying Yang, Minh Hoai Nguyen,Margara Tejera Padilla, Feng Zhou, and Fernando De la Torre, Detecting Depression from Facial Actions and Vocal Prosody.

23. Kenneth Hoffman and Ray Kunze, Linear Algebra, Second edition, 1971.

24. Marcus Marvin, Minc Henrynk, Introduction to Linear Algebra, published by Dover Publications, Mineola, New York, U.S.A.,1988.

25. Dr.S.S Dinakaran, A.Ruby, A New Approach to Identify the Stegno Image Using Eigen Value and Eigen Vector Method, International Journal of Engineering Trends and Technology- Volume3Issue3- 2012 , 2231-5381