A Literature Review : Effect of Plastic Surgery on Face Recognition

A Literature Review : Effect of Plastic Surgery on Face Recognition

Minal Mun

M. Tech. Scholar, Department of Computer Science and

Engineering, Government College of Engineering, Amravati, Maharashtra, India.

Prof. Anil Deorankar

Associate Professor, Department of Computer Science and Engineering, Government College of Engineering, Amravati, Maharashtra, India.

Dr. Prashant Chatur Head of Department, Department of Computer Science and Engineering, Government College of Engineering, Amravati Maharashtra, India

Abstract

Variation in pose, expression, illumination, occlusion and aging are the major problem in face recognition and algorithms have been proposed to handle these challenges. Except this new problem in face recognition is plastic surgery. This problem remains still less explored topic in face recognition domain. This paper focuses on analyzing the effect of plastic surgery in face recognition algorithms. Also explain the reason for plastic surgery and various types of facial surgery due to which textural as well as shapial feature of the face will change and degrade the performances of face recognition algorithm. Therefore, it is imperative for future face recognition systems to be able to address this important issue and hence there is a need for more research in this important area.

1. Introduction

   Recently, technology became available to allow verification of "true" individual identity. This technology is based in a field called "biometrics". Biometric access control are automated methods of verifying or recognizing the identity of a living person on the basis of some physiological characteristics, such as fingerprints or facial features, or some aspects of the person's behavior, like his/her handwriting style or keystroke patterns. Since biometric systems identify a person by biological characteristics, they are difficult to forge. Among the various biometric ID methods, the physiological methods (fingerprint, face, DNA) are more stable than methods in behavioral category (keystroke, voice print). The reason is that physiological features are often non-alterable except by severe injury. The behavioral patterns, on the other hand, may fluctuate due to stress, fatigue, or illness. However, behavioral IDs have the advantage of being nonintrusiveness. People are more comfortable signing their names or speaking to a microphone than placing their eyes before a scanner or giving a drop of blood for DNA sequencing. Face recognition is one of the few biometric methods that possess the merits of both

   high accuracy and low intrusiveness. Also, it provides information about Age, gender, personal identity (physical structure), Mood and emotional state (facial expression) and Interest / attentional focus (direction of gaze). However, even after decades of research, face is still an active topic because of the variability observed in face due to illumination, pose, expression and occlusion. A new challenge to face recognition is facial plastic surgery. These surgery alters the facial features to such an extent that humen being often struggle to identify a person face after surgery. The figure 1 shows an example of the effect of plastic surgery on facial appearances.

   Figure1. The effect of plastic surgery on facial appearances

   Popularity of plastic surgery has increased many folds over the past few years and the statistical data shows that it keeps growing[6]. Due to advances in technology, affordability, and the speed with which these procedures can be performed, several people undergo plastic surgery for medical reasons and some choose cosmetic surgery to look younger or for better appearance. The procedures can significantly change the facial regions both locally and globally, altering the appearance, facial features and textur.

   Again, due to privacy issues, the surgical details of a particular individual are not available and plastic surgery face database contains one pre-surgery image for training and a post-surgery image for testing. This further complicates feature extraction task in face recognition methods. Also, Each facial plastic surgery changes shape or texture of a particular face region. It

   is very difficult to predict which features are invariant (a region without surgery effects) with unavailable surgery information. The difficulty is further supplemented, when an individual undergoes more than a surgery. The existing face recognition algorithm are good in extracting one of feature from an image i.e. either shape or texture[5].

   The plastic surgery can also be misused by individuals who are trying to conceal their identity with the intent to commit froud or evade law enforcement. Also this surgery allow the theft or terrorist to freely move around without any fear of being identified by any face recognition system.again it might lead to rejection of genuine users.

   So it is necessary to develop a method for face recognition under plastic surgery.

2. Related work

   Traditionally, face recognition research has focused primarily on developing novel characterizations and algorithms to deal with challenges posed by variations in acquisition conditions like illumination conditions and head pose with respect to the camera. Tremendous success in dealing with these problems is probably one of the primary factors that has generated interest in new avenues in face matching that include matching faces across plastic surgery variations.

   1. A sparse representation approach :

      Gaurav Aggarwal, Soma Biswas, Patrick J. Flynn and Kevin W. Bowyer[3], proposed a novel approach to address the challenges involved in automatic matching of faces across plastic surgery variations. In the proposed formulation, they proposed a part-wise sparse representation Approach combined with the popular sparse representation to address the challenge of plastic surgery variations and utilizes images from sequestered non-gallery subjects with similar local facial characteristics to fulfill this requirement. They stated that this sparse representation approach also used for several other biometrics and computer vision problems. One limitation of sparsity-based biometric recognition is, it requires several images per subject in the gallery.

   2. Near set theory approach :

      K. R. Singh, Roshni S Khedgaonkar, Swati P Gawande [4], proposed a new approach to find the nearness between the pre plastic surgical face to the post plastic surgical face. They develop a classifier for facial images that have previously undergone some feature modifications through plastic surgery based on near set theory. Their work concerned only geometrically obtained feature values and their approximation using near sets. Once the features will

      be extracted a feature database will be formed. Using this feature values near set theory provides a method to establish resemblance between objects contained in a disjoint set, that is it provides a formal basis for observational comparison and classification of the objects. One limitation to this approach is, it will recognize the face only after local plastic surgery, but not work in the presence of global plastic surgery.

   3. Multiobjective evolutionary approach :

      Himanshu S. Bhatt, Samarth Bharadwaj, Richa Singh, and Mayank Vatsa [1], proposed a multiobjective evolutionary granular algorithm to match face images before and after plastic surgery. The algorithm first generates non-disjoint face granules at multiple levels of granularity. The first level of granularity processes the image with Gaussian and Laplacian operators to assimilate information from multiresolution image pyramids. The second level of granularity tessellates the image into horizontal and vertical face granules of varying size and information content. The third level of granularity extracts discriminating information from local facial regions. After feature is ectracted from that face granules by SIFT and EUCLBP algorithm. Then Multiobjective Evolutionary Approach is use to optimization of weight. Decision is take place on the basis of weight

3. Face recognition algorithm

   There are various face recognition algorithm such as PCA, FDA, LFA, Local Binary Pattern and Neural Network which are invariant to illumination, pose and expression[6][8]. The effects of plastic surgery on this algorithm are explained as follows.

   1. Principal component analysis:

      PCA is a appearance-based algorithms which is used to form feature vector and dimensionality Reduction. PCA yields 59.3% identification accuracy when using the non-surgery database (face images with neutral expression, proper illumination, and no occlusion). On the other hand, the accuracy decreases by 30% when evaluated with pre- and post-surgery face images.

   2. Fisher discriminant analysis:

      FDA is also appearance-based algorithm which is used for face recognition. The accuracy of FDA on non-surgery face database is 61.6% while on plastic surgery face database is 32.5% which is not acceptable in real-world applications.

   3. Local feature analysis:

      LFA is a feature based algorithm in which the feature is extracted from lacal part of face like nose,

      mouth, eye etc. The accuracy of LFA on non-surgery face database is 68.9% while on plastic surgery face database is 38.6%

   4. Speeded up robust features:

      SURF is a descriptor-based approach which is also used for face recognition. The accuracy of SURF on non-surgery face database is 77.7% while on plastic surgery face database is 50.9% which is very less.

   5. Local binary pattern:

      Local Binary Patterns[9] provide a powerfulmeans of texture description. LBP features are gray scale and rotation invariant texture operator. These features are more widely used for expression recognition. LBP features are also applied for face recognition task. LBP feature extraction is faster than any other feature extraction method and it provides good performance make this most researched features. The accuracy of LBP on non- surgery face database is 73.6% while on plastic surgery face database is 47.8%.

   6. Neural network architecture based 2-d log polar gabor transform:

      Local features in face images are more robust against distortions such as pose, illuminations etc, and a spatial-frequency analysis are often desirable to extract such features. With good characteristics of space-frequency localization, Gabor wavelet analysis is a suitable choice for face recognition purpose. An image can be represented by Gabor wavelet responses by convolving Gabor filters of different scale and orientation. The set of convolution coefficients for kernels at one image pixel is called a jet. The resulting output contains most important face features like eyes, mouth and nose edges, as well as moles, dimples and scars. Magnitude information of convolved face image is preferred because it makes data invariant under rotation or translation. The accuracy of Gabor on non- surgery face database is 84.1% while on plastic surgery face database is 54.2%

      The above comparison shows that plastic surgery is a very challenging problem to face recognition other than illumination or expression and hence the development of algorithms to confound these effects is required.

4. Challenges of face recognition after plastic surgery

   Facial plastic surgery changes face appearance, which intuitively affects the robustness of appearance based face recognition. In this section, we analyze the effects of different plastic surgery procedures on face appearance

   1. Changes in skin texture:

      Some plastic surgery makes people look younger or more attractive by removing face scars, acnes or taking skin resurfacing. As a result, the skin texture will change.

   2. Changes of face component:

      The main face components: forehead, eyelid, nose, lip, chin and ear can be reshaped or restructured by plastic surgery. The local skin texture around the face component may also be disturbed.

   3. Changes of global face appearance:

      Global facial plastic surgery will change the global face appearance, in other words, not only part of the face component and the skin texture will change, but also the whole face geometric structure and appearance will be disturbed.

      In summary, the challenges of face recognition after plastic surgery mainly lie in the fact that faces after plastic surgery have undergone various appearance changes, but no method is available to detect or model such changes.

5. Types of facial plastic surgery

   When an individual undergoes plastic surgery, the facial features are reconstructed either globally or locally[6][8]. Therefore, in general, plastic surgery can be classified into two distinct categories.

   1. Disease correcting local plastic surgery (Local surgery):

      This is a kind of surgery in which an individual undergoes local plastic surgery for correcting defects, anomalies, or improving skin texture. Local plastic surgery techniques can be applied for possibly three different purposes: 1) to correct by-birth anomalies, 2) to cure the defects that are result of some accident, and

      3) to correct the anomalies that have developed over the years. Examples of disease correcting local plastic surgery would be surgery for correcting jaw and teeth structure, nose structure, chin, forehead and eyelids etc. Local plastic surgery is also aimed at reshaping and restructuring facial features to improve the aesthetics. This type of local surgery leads to varying amount of changes in the geometric distance between facial features but, the overall texture and appearance may look similar to the original face. However, any of the local plastic surgery procedures may be performed in conjunction with one or more such procedures and an amalgamate of such procedures may result in a fairly distinct face when compared to the original face.

   2. Plastic surgery for reconstructing complete facial structure (Global surgery):

      Apart from local surgery, plastic surgery can be performed to completely change the facial structure which is known as full face lift. Global plastic surgery is recommended for cases where functional damage has to be cured such as patients with fatal burns or trauma. Note that, global plastic surgery is primarily aimed at reconstructing the features to cure some functional damage rather than to improve the aesthetics. In this type of surgery, the appearance, texture and facial features of an individual are reconstructed to resemble normal human face but are usually not the same as the original face. Furthermore, global plastic surgery may also be used to entirely change the face appearance, skin texture and other facial geometries making it arduous for any face recognition system to recognize faces before and after surgery. Therefore, it can also be misused by criminals or individuals who want to remain elusive from law enforcement and pose a great threat to society despite all the security mechanism in-place.

      In the above mentioned categories of facial plastic surgery, there are several types of surgeries which are described as follows:

      1. Rhinoplasty (nose surgery): It is used to reconstruct the nose in cases involving birth defects, accidents where nose bones are damaged and also to cure breathing problems caused due to the nasal structure. Cosmetic Rhinoplasty is used for those who wish to straighten or narrow their nose to improve their facial appearance. It is also used to prevent the nose structure deformation due to aging.

      2. Blepharoplasty (eyelid surgery): Eyelid is the thin skin that covers and protects our eyes. Blepharoplasty may be used to reshape both upper as well as lower eyelid in cases where excessive growth of skin tissues on the eyelid causes vision problem.

      3. Brow lift (forehead surgery): It is generally recommended for patients above the age of 50 who suffer from flagging eyebrows (due to aging) which obstruct vision. It is also helpful in removing thick wrinkles from the forehead and giving a younger look.

      4. Genioplasty/Mentoplasty (chin surgery): It is mostly used to reshape the chin including smooth rounding of the chin, correcting bone damages, and reducing/augmenting chin bones.

      5. Cheek implant: It is used to improve the facial appearance and it can be divided into two classes, malar and sub-malar augmentation. In malar augmentation a solid implant is fitted over the cheek

         bone whereas in sub-malar augmentation implants are fitted in the middle of the cheeks where the person has a recessed (hollow) look.

      6. Otoplasty (ear surgery): It involves bringing the ears closer to the face, reducing the size of ears and orienting/pruning some structural ear elements.

      7. Liposhaving (facial sculpturing): It is a technique used to get rid of the excess fat attached to the skin surface on the face, especially in chin and jaw regions. This technique is commonly used to remove the dual chin that grows because of surplus fat below the chin.

      8. Skin resurfacing (skin peeling): There are different techniques such as laser resurfacing and chemical peel to treat wrinkles, stretch marks, acne and other skin damages caused due to aging and sun burn. Skin resurfacing results in smooth skin with ameliorated texture.

      9. Rhytidectomy (face lift): It is used to treat patients with severe burns on face and neck. Face lift surgery can also be employed to fight aging and get a younger look by tightening the face skin and thus minifying wrinkles.

      10. Lip augmentation: Lips have a pronounced role in an individuals beauty. Cosmetic surgery for lip augmentation involves proper shaping and enhancement of lips with injectable filler substances.

      11. Craniofacial: This type of surgery is employed to treat by-birth anomalies such as Clift lip and palate (a gap in the roof of mouth), microtia (small outer ear) and other congenital defects of jaws and bones. Some defects may be treated soon after birth but for some (like microtia), the patient may have to wait up to an age of 10-14 years.

      12. Dermabrasion: It is used to give a smooth finish to the face skin by correcting the skin damaged by sun burns or scars (developed as a post surgery effect), dark irregular patches (melasma) that grow over the face skin and mole removal.

      13. Non-surgical procedures: There are several non-surgical procedure for skin resurfacing, wrinkle removal, and acne/scars removal. For example, laser resurfacing for acne scars, photodynamic therapy or photo-rejuvenation treatments, and BOTOX or filler injections.

          Among all the techniques listed above Rhinoplasty, Blepharoplasty, Forehead surgery, cheek implant,

          Otoplasty, Lip augmentation, and Craniofacial are purely local surgeries. On the other hand, Rhytidectomy (face lift) is purely global plastic surgery whereas Liposhaving, Skin resurfacing, and Dermabrasion can be both local and global. In order to protect the identity of the individuals, if possible, only the local facial features that are reconstructed are shown and not the complete face. These procedures usually alter the position of key fiducial points, thus changing the overall appearance of the face. This, in effect, leads to reduced performance of face recognition algorithms. The techniques that modify key fiducial points such as nose, forehead, chin, eyelid, eyebrows, mouth and lips have a more pronounced effect on face recognition systems than the techniques which deal with ears, mole removal, and Dermabrasion.

6. Conclusion

   Current face recognition algorithms mainly focus on handling pose, expression, illumination, aging and disguise. This paper formally introduces Plastic surgery, which alter the various features of humen face, is a new challenge to face recognition algorithms. In this paper, we present different face recognition algorithms and their performance on a plastic surgery database that contains face images with both local and global surgeries. The study shows that PCA, FDA, GF, LFA, LBP and GNN algorithms are unable to effectively mitigate the variations caused by the plastic surgery procedures. Also we reviewed the various Challenges to face recognition algorithm after plastic surgery and types of facial plastic surgery. Based on the results, we believe that more research is required in order to design an optimal face recognition algorithm that can also account for the challenges due to plastic surgery. So it is necessary to develop a method for face recognition which is invariant to plastic surgery.

7. References

1. Himanshu S. Bhatt, Samarth Bharadwaj, Richa Singh, and Mayank Vatsa, Recognizing Surgically Altered Face Images Using Multiobjective Evolutionary Algorithm, IEEE Transactions On Information Forensics And Security, Vol. 8, No. 1, January 2013

2. P.Fasca Gilgy Mary, P.Sunitha Kency Paul, J.Dheeba

   ,Human Identification Using Periocular Biometrics, in International Journal of Science, Engineering and Technology Research (IJSETR) Volume 2, Issue 5, May 2013

3. Gaurav Aggarwal, Soma Biswas, Patrick J. Flynn and Kevin W. Bowyer,A Sparse Representation Approach to Face Matching across Plastic Surgery IEEE Workshop on Applications Of Computer Vision (WACV), 2012

4. K. R. Singh, Roshni S Khedgaonkar, Swati P Gawande

   A New Approach to Local Plastic Surgery Face Recognition Using Near Sets, in International Journal of Engineering Science and Technology, NCICT Special Issue, Feb 2011

5. N. S. Lakshmiprabha, J. Bhattacharya, and S. Majumder, Face recognition using multimodal biometric features, in International Conference on Image Information Processing. IEEE, 2011, pp. 16.

6. R. Singh et al., Plastic surgery: A new dimension to face recognition, IEEE Transaction On Information Forensics and Security, vol. 5, no. 3, pp. 441448, 2010.

7. D. Woodard, S. Pundlik, J. Lyle, and P. Miller,

   Periocular region appearance cues for biometric identification, in CVPR Workshop on Biometrics. IEEE, 2010, pp. 162169.

8. M. Singh, R. Vatsa and A. Noore, Effect of plastic surgery on face recognition: A preliminary study, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops. IEEE, 2009, pp. 7277.

9. Di Huang, Caifeng Shan, Mohsen Ardabilian, Yunhong Wang, and Liming Chen, Local Binary Patterns and Its Application to Facial Image Analysis: A Survey, IEEE Transactions On Systems, Man, And CyberneticsPart C: Applications And Reviews, Vol. 41, No. 6, November 2011.

10. Sangeeta Kakarwal, Ratnadeep Deshmukh ,Wavelet Transform based Feature Extraction for Face Recognition, in International Journal of Computer Science and Application Issue 2010

11. Harine Sellahewa, Sabah A. Jassim, Image Quality Based Adaptive Face Recognition, IEEE Transaction On Instrumentation And Measurement, Vol. 59, No. 4,

    April 2010

12. Hassen Drira, Boulbaba Ben Amor, Anuj Srivastava, Mohamed Daoudi, Rim Slama,3D Face Recognition Under Expression, Occlusions, And Pose Variations.,IEEE Trasaction On Pattern Analysis And Machine Intelligence, Vol. 35, No. 9, September 2013

13. Zahid Riaz, Michael Beetz, Bernd Radig,Shape Invariant Recognition of Segmented Human Faces using Eigenfaces, International Conference on Multitopic Conference IEEE 2008