Hybrid Approach for Facial Feature Extraction

Hybrid Approach for Facial Feature Extraction

Amit Kaul

Electrical Engineering Department National Institute of Technology Hamirpur

Hamirpur, H.P., India.

Sushil Chauhan

Electrical Engineering Department National Institute of Technology Hamirpur

Hamirpur, H.P., India.

A. S. Arora

Electrical and Instrumentation Department,

Sant Longowal Institute of Engineering & Technology, Longowal, Sangrur, Punjab, India

Abstract Human face in addition to revealing the identity of an individual also provides information regarding his/her emotional and psychological state. In addition to this various facial features are utilized in applications like biometrics (face, iris and lips), automatic lip-reading, speech recognition etc. For accomplishing these tasks reliable face and facial feature detection methods are required. In this work a hybrid facial feature extraction approach based on wavelets and information from HSV and YCbCr color spaces has been presented. The main emphasis of this paper is to extract lip-region for further applications. The algorithm has been tested on images captured by different resolution cameras under varying illumination condition.

Keywords Wavelet, color spaces, lip localization

1. INTRODUCTION

   Unconsciously in everyday life humans effortlessly capture tremendous amount of information from faces of one another such as recognizing each other, guessing emotional and health state of an individual while interacting with him or her. This wealth of information present in faces is utilized in numerous applications present in commercial and forensic domain like lip-reading, speech recognition, face biometrics etc. These applications in majority of the cases are based on facial features. For development of these processes keeping human machine perspective in mind reliable facial feature (eyes, nose and mouth or lips) detection mechanism is required. Robust facial feature detection is a challenging task because of the variability arising in images due to change in expression, gender, occlusion, illumination, pose etc.

   In this paper a hybrid approach is presented which utilizes the complementary information present in the wavelet domain and different color spaces for detection of face and extraction of three main facial features. The major emphasis has been on the localization of the lip region so as to utilize it for further applications like lip reading and lip based biometrics.

   The paper is organized as follows. This first section provides the introduction. Brief overview of methods employed for facial feature detection and applications that utilize information from lip region is presented in the next section. The proposed method is described in section III.

   Experimental results are shown in section IV and finally paper is summarized with a brief conclusion.

2. RELATED WORK

   In order to extract the facial features face region is identified in a given image. Face detection has been carried out using wavelet and color spaces [1, 2]. Extraction of facial features particularly lip localization is a difficult task. In general techniques utilized for solving this problem can be grouped under three headings (a) image based methods comprising of color based methods and subspace methods, (b) model based methods and (c) hybrid methods.

   Lip prints have been extensively studied in dental forensics [3, 4] but machine based lip print recognition is relatively new area. L. Smacki and et. al have suggested the use of Hough transform for automatic lip print recognition using scanned lip-print images [5]. S. Bhattacharjee and his fellow researchers have used canny edge detector to generate region specific edge datasets for identifying four individuals [6]. E. Gomez and et.al on the other hand have used lip shape based features such as height and width of lips and polar coordinates of lip envelope for biometric identification [7]. M. Choras has also employed lip shape features like geometrical parameter of lip area, ratio of lower and upper lip height etc. and lip color features (maximum, minimum, mean intensity etc.) for human identification [8, 9].

   As far as human recognition using digital images acquired from camera is concerned K. S. Baik and his colleagues have proposed a scheme in which multi-resolution architecture is used for person identification. The local pattern masks in horizontal vertical and diagonal direction are used to find the similar pattern in input image. Based on the statistical factors (mean and standard deviation) feature vector is obtained [10].

3. METHODOLOGY ADOPTED

   Facial feature extraction first requires localization of the face in a given image. After this one of the key points out of eyes, nose or lips are detected and considering this as reference other features are located [11, 12, 13].

   In order to highlight the edge information of different facial features properties of two-dimensional wavelet transform are exploited [14.15]. Two dimensional wavelet transform uses a two dimensional scaling function (x, y) and

   three two-dimensional wavelets, H (x,y), V (x,y) and D (x,y) [19,22]. The two-dimensional wavelet decomposition of an image I0(m,n) on J octaves results in 3J+1 sub-images

   [, {, , } ]

   1. Using K-means clustering the pixels in localized face for which Cb and Cr values were in the range mentioned in step 2 were segregated into two bins. The centroid value of each bin is treated as the location of

      =1,,

      the eyes.

      j

      j

      j

      j

      where aJ is the low resolution approximation of the original image and the d k are the wavelet sub-images containing the image details at different scales (2j) and orientations (k). Wavelet coefficients d Hj , d V , and d Dj respectively provide a measure of intensity variations in horizontal, vertical and diagonal direction.

      1. Algorithm

         The main steps of the approach adopted in this work are given below:

         1. Convert given input image (, ) in RGB color space to get HSV and YCbCr color space images

            (, ) and (, ) respectively. Modified value of hue (, ),was also computed by using the following expression:

            h m, n R m, n

            G(m, n) B(m, n)

         2. Compute wavelet decomposition of grayscale image of the input image using Haar wavelet up to level three.

         3. All regions for which Cb and Cr values of IYCbCr(m, n) are outside range [100 130] and [130 150] respectively are masked in horizontal sub-image obtained above after wavelet decomposition in step 2. Localize the face by selecting blocks in the horizontal sub-image for which mean value was above a given threshold.

   1. For locating nose, region below location of eyes at a distance of 1/3rd of total face length is tracked. Midpoint of the largest horizontal edge in this region is considered nose.

   2. Below the location of nose, pixels for which value of hue >0.019, saturation <0.040 and modified hue>0.6 were extracted. The pixels corresponding to these values formed the lip contour.

   The entire process of detection of eyes nose and finally extraction of lips has been depicted in Figure 1.

4. EXPERIMENTAL RESULTS

   The proposed algorithm was tested on an in-house database comprising of images collected under varying illumination conditions and with different resolution cameras. Also the distance of the subjects from the camera was variable. For well lighted surroundings three images were obtaied for fourteen individuals. Another set of images were collected under two different illumination settings but with lesser number of subjects. In total testing was carried on 105 images.

   Figure 2 depicts the facial feature extraction results for the images of the same subject under different illumination conditions. Finally the results of extracted lips for different subjects are shown in figure 3.

   Input Image Masked horizontal sub-image Localized Face

   Extracted lip

   Facial feature detection

   Eyes detection

   Fig.1: Flow diagram for lip extraction

   Fig.2: Examples of facial feature detection under varying illumination

   Fig.3: Examples of lips extracted for three subjects

5. CONCLUSIONS

The approach adopted in this work has shown promising results in extracting facial feature from images collected under different illumination conditions. However the suggested method is suitable for frontal pose still images of single person. Future work in this direction will be to extend this approach for images with more than one face and use the extracted facial features for biometric applications.

REFERENCES

1. P. Viola, and M.J. Jones. "Robust real-time face detection."International journal of computer vision 57.2 (2004): 137- 154.

2. S. K. Singh, D. S. Chauhan, M.Vatsa,, & R.Singh, (2003). A robust skin color based face detection algorithm. Tamkang Journal of Science and Engineering, 6(4), 227-234.

3. T. R Saraswathi,, G. Mishra, and K. Ranganathan. "Study of lip prints."Journal of forensic dental sciences 1, no. 1 (2009): 28.

4. R. Venkatesh, , and M. P. David. "Cheiloscopy: An aid for personal identification." Journal of Forensic Dental Sciences 3, no. 2 (2011): 67- 70.

5. L. Smacki., Piotr Porwik, Krzysztof Tomaszycki, and S. Kwarcinska. "The lip print recognition using Hough transform." Journal of Medical Informatics and Technologies 14 (2010): 31-38

6. S. Bhattacharjee, , S. Arunkumar, and Samir Kumar Bandyopadhyay. "Personal Identification from Lip-Print Features using a Statistical Model."International Journal of Computer Applications 55 (2012).

7. E. Gomez, C. M. Travieso, J. C. Briceno, and M. A. Ferrer. "Biometric identification system by lip shape." Proceedings of 36th Annual 2002 International Carnahan Conference on Security Technology, 2002., pp. 39-42. IEEE, 2002.

8. M. Chora,. "Human lips as emerging biometrics modality." Image Analysis and Recognition, pp. 993-1002. Springer Berlin Heidelberg, 2008.

9. M..Chora, "The lip as a biometric." Pattern Analysis and Applications 13, no. 1 (2010): 105-112.

10. J. O. Kim, , W. Lee, J. Hwang, K. S. Baik, and C. H. Chung. "Lip print recognition for security systems by multi-resolution architecture." Future Generation Computer Systems 20, no. 2 (2004): 295-301.

11. Du, Hua Gu Guangda Su Cheng. "Feature points extraction from faces." Image and Vision Computing NZ, Palmerston North (2003): 154-158.

12. C. C. Chiang, , W. K. Tai, M. T. Yang, Y.T. Huang, and C. J. Huang. "A novel method for detecting lips, eyes and faces in real time."Real- time imaging 9, no. 4 (2003): 277-287.

13. M. Li, and Y. M. Cheung. "Automatic lip localization under face illumination with shadow consideration." Signal Processing 89, no. 12 (2009): 2425-2434.

14. Gonzael, R.C., and Woods, R.E., 2005, Digital Image Processing, Pearson Publication 2nd Edition.

15. M. Tico, E. Immonen, P. Kuosmanen Ramo, P. and J. Saarinen, 2001. Fingerprint recognition using wavelet features. In proceedings of International Symposium on Circuits and Systems.