Human Face Image Retrieval and Verification by Orthogonal Matching Pursuit Algorithm

Human Face Image Retrieval and Verification by Orthogonal Matching Pursuit Algorithm

Krishna Chandran,

1. Scholar

   Anusmina D. J., Asst.Professor

   Dept. of ECE Dept. of ECE

   TKM Institute of Technology TKM Institute of Technology Kollam, India Kollam, India

   Abstract–Face image retrieval aims to retrieving facial images that are relevant to users requests from a large dataset of images. To improve the efficiency of the retrieval system, our system uses a combination of two types of features. One feature is based on the texture component of the image and other feature is skin colour of face images. Combination of these two level features improves the speed and efficiency of proposed scheme. In training stage, skin colour based sparse words are find out for all images in data base. In testing, a comparison with input data is performed.

   Key WordsTexture; Sparse words; Local Binary Patterns; Local image patches

   1. INTRODUCTION

      Face image retrieval [1]has many applications in so many areas such as Image tagging, Crime investigation, etc. So there are a number of methods are used for such applications. When we use a specific feature for image retrieval,system efficiency become very low.

      In this work, we use a combination of two types of features. One of the feature acts as afiltering mechanism to improve system accuracy. Skin colour of a person does not change over his life time. So when we use skin colour as a filtering element, efficiency of the system improves.

      Textures of images are taken as the other feature. For texture based feature Local Binary Pattern of images are find out. Instead of finding LBP of the whole face, we take the LBP of facial components and combine it.

      The significance of using face colour information in our system is: give more descriptive evidence for similarity or dissimilarity, improve the speed of the system and improve accuracy.

      Recent days sparse coding has better results on many different applications such as image compression, noise reduction, feature extractionetc. Taking advantage of sparse coding, this system introduces anefficient face retrieval scheme. When face image retrieval is performed with skin colour information, an input image able to get most similar images from the dataset with high efficiency.

   2. RELATED WORKS

      Our work is closely related to several researches of face detection and retrieval. For face detection a neural network-based upright frontal face detection systemis used

      in the earlier stages. Here aretinally connected neural network examines small windows of an image, and decides whether each window contains a face or not. If the window consists of face, it give output +1 and otherwise it give output as -1.

      Automatic human face image detection systemanother method and is based on the detection of shape features and skin colour [8].

   3. SPARSE CODING WITH SKIN COLOUR

      In this section, sparse words for face images are find out. Local binary pattern is given as the input of sparse representation block. Skin colour helps to improve the efficiency of the system.

      Fig. 1.Block diagram

      For any face image retrieval or recognition system, face detection is the first and important step. Face detection has several applications in the areas like human- computer interface, videocoding, conferencing etc. For face detection in our system, we apply Viola- Jones algorithm [7]. It is simple and efficient. We also detect Eyes, Nose and Mouth

      patches in the face and extract them for finding the LBP features [5]. In our system, LBP of different patches are find out and a combination of patch level LBP is used as a featurevector for a specific face. This LBP is used to obtain sparse codewords of face. Skin colour of the face is also given as side information. Sparse words of all images in the database are find out in the training stage of the system. In the testing stage, when an input image arrives all the steps are carried out in the input image and a comparison between sparse words are performed. It is done by measuring the hamming distance between these sparse words. The images with similarity less than a threshold value is retrieved and displayed.

      A.Skin Colour Detection

      For detecting skin colourfirst step is to make a skin model [6]. The images are converted from RGB to YCbCrcolourspace. And the resulted image is then converted in to a binary image by applying some thresholds. Pixels lying in the threshold range were made white and the others were made black. Then the output is a binary image with 1s representing skin pixels and 0s representing non- skin pixels. Then apply morphological operations such as filling, erosion and dilation in order to separate the skin areas which are loosely. Morphological closing is applied firstly to the binary image. Dilated binary image is multiplied with binary image from segmentation process to maintain the holes. This is due to later stage which will use number of holes to filter out some non-face regions. Output image is a mask image.

      In the final stage crosscorrelation between template face and grayscale region is taken. By taking an appropriate threshold value images are classified into white or black.

      B.Local Binary Pattern Features

      LBP features are find out to given as the input of sparse coding block [5]. LBP is a simple and effective feature, widely used for texture description. For finding LBP feature of an image, first divide the image into small regions and for each region separately LBP features are find out. For a 3×3 image region, each pixel is compared with its central value. If the pixel is greater than the center value, replaced it with 1 and otherwise by 0. So we get a binary string as output. Corresponding decimal value is taken and replace the center value with it.

      Fig. 2.Applying LBP in an image

      C. Sparse coding

      Sparse representation is based that, images can beapproximately represented as a linear combination of some basic vectors from a dictionary. For representing an image in sparse words, only few codewords are required. That means, any signal or image with a large size can be represented with small set of wordswithoutlosing the information.

      Generally used algorithms to solve sparse coding are, Matching pursuit(MP) algorithm and Orthogonal matching pursuit(OMP) algorithm. OMP is the orthogonal version of MP algorithm and it is used in our system. Here we use DCT as our dictionary.

      In OMP algorithm, we take the LBP of face images and created dictionary as the input. And also take a stopping condition based on a threshold. Our output is a list of coefficients. When we find sparse representation of image, most of the code words are zeros and non-zero entries are taken as the code words of the input image.

      In the first step of OMP algorithm, take LBP as initial residual function R(0). Find k*, which is the maximum inner product between a dictionary element and the residual. For obtaining the sparse representation, take the innerproduct between the residual and the element corresponding to the k*th position in the dictionary. Update the residual value and repeat the steps until the stopping condition.

      D.Sparse Representation using Skin colour

      For considering skin colour in the sparse representation; first divide dictionary to obtain different sparse code for images with different attributes [9]. For a single human attribute; divide dictionary into two different subsets. Images with positive attribute score will use one of the subset and images with negative attribute score will usethe other. Images with white face colour, we use first half of the dictionary and for black faces, second half od the dictionary elements are used.

   4. EXPERIMENTAL RESULTS MATLAB is a numerical computing environment and

      fourth-generation programming language and here

      MATLAB R2013a is used as simulation tool. Images from LFW dataset is used for experiments. In LFW data set, each face has been labeled with the name of the person pictured.

      As a part of this project, Face detection is carried out with Viola Jones face detector. Local image patches are extracted and local binary pattern feature is also obtained. As an attribute the faces are also classified based on skin colour. Simulation is done in MATLAB R2013a. The simulation results are given below.

      1. Input image

         Figure 8 shows the input imagehaving the size 150 x 150 x 3 (from LFW database).

         Fig. 3. Input image

      2. Results

      For face detection first viola jones face detector is applied to the input image. Detected face is shown in Figure

      4. Detected eye is shown in Figure 6, detected nose is shown in Figure 8 and detected mouth is shown in Figure 10.

      Fig. 4. Detected face Fig. 5. Extracted face

      Fig. 6.Eye detection Fig. 7. Extracted eye

      Fig. 8.Nose detection Fig. 9. Extracted nose

      Fig. 10.Mouth detectionFig. 11. Extracted Mouth

      For the extracted local face patches patch level local binary patterns are find out. Patch leval LBP features are shown in Figures.

      Fig. 12. Face patch, LBP, Histogram of LBP

      Fig. 13.Eye patch, LBP, Histogram of LBP

      Fig. 14.Mouth patch, LBP, Histogram of LBP

      Fig. 15.Nose patch, LBP, , Histogram of LBP

      For detecting face is black or white skin segmented image is find out. From the input gray image and template, face region is extracted. Find average gray value and compare with a threshold. For the input image, skin segmented image and template matched output is shown in Figure 16.

      image comes, its sparse code words are also find out and retrieve images based on similarity. Face verification is also used to verify the result.

      Fig. 16.Skin Segmented image and template matched output

      Fig. 17.Output in command window

      Take another image to detect skin colour. First skin segmentation is carried out and template matched output also find out. Outputs are shown in Figure 19 and Figure 20 respectively.

      Fig. 18.Input image for attribute detection

      Fig. 19.Skin Segmented image and template matched output

      Fig. 20.Output in command window

   5. CONCLUSIONS

First step of all face retrieval and recognition system is face detection. Here face detection is carried out by Viola Jones face detection algorithm. Itminimizes computation time while achieving high detection accuracy. For improving retrieval results facial colourbased classification also used with low level features.Local binary pattern is a powerful means of texture description. By LBP face images can be seen as a composition of micro- patterns. It is a simple and efficient representation of face images. In this approach a face image is first divided into several regions.From which extract local binary patterns and construct a global feature histogram that represents both the statistics of the facial micro-patterns and their spatial locations.By using LBP and with the help of skin colour, sparse codewords are constructed and stored. When an input

REFERENCES

1. Bor-Chun Chen, Yan-Ying Chen,Yin-HsiKuo, Winston H. Hsu, Scalable Face Image Retrieval using Attribute Enhanced Sparse Codewords, in IEEE Transactions on Multimedia Vol:PP no:99 year 2013.

2. N. Kumar, A.C. Berg, P.N Belhumeur, and S.K. Nayar, Describable visual attributes for face verification and image search, in IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Special Issue on Real-World Face Recognition, Oct 2011.

3. Z.Wu, Q.Ke, J. Sun, and H.-Y. Shum, Scalable face image retrieval with identity-based quantization and multi-reference re-ranking, IEEE Conference on Computer Vision and Pattern Recognition, 2010.

4. U. Park and A. K. Jain, Face matching and retrieval using soft biometrics, IEEE Transactions on Information Forensics and Security, 2010.

5. T. Ahonen, A. Hadid, and M. Pietikainen, Face recognition with local binary patterns, European Conference on Computer Vision, 2004.

6. Rein-Lien Hsu, Abdel- Mottaleb, and A. K. Jain, Face Detection in Colour Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, May 2002.

7. P.Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, IEEE Conference on Computer Vision and Pattern Recognition, 2001.

8. Peter Peer Franc Solina, An Automatic Human Face Detection Method, Proc. Of Computer Vision Winter Workshop, 2000

9. M. Douzeand A. Ramisa and C. Schmid, Combining Attributes and Fisher Vectors for Efficient Image Retrieval, IEEE Conference on Computer Vision and Pattern Recognition, 2011.

10. D. Wang , S.C Hoi Y. He, and J. Zhu, Retrieval-based face annotation by weak label regularized local coordinate coding, ACM Multimedia, 2011.

11. J. MairalF. Bach, J. Ponce, and G. Sapiro, Online dictionary learning for sparse coding, ICML, 2009.

12. A. Coates and A. Y. Ng, The importance of encoding versus training with sparse coding and vector quantization, ICML, 2011.

13. J.Wang , J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong, Locality constrained linear coding for image classification, IEEE Conference onComputer Vision and Pattern Recognition, 2010.