An Approach for Removal of Occlusion using Examplar based Image-Inpainting Technique: A Review

An Approach for Removal of Occlusion using Examplar based Image-Inpainting Technique: A Review

Sandesh V Khamele1

1M.Tech Scholar, CSE Department Ramdeobaba College of Engineering and management

Nagpur, India

S. G. Mundada2

2Assistant Professor, CSE Department Ramdeobaba College of Engineering and management

Nagpur, India

Abstract Systems that rely on Face Recognition (FR) biometric have gained great importance ever since terrorist threats imposed weakness among the implemented security systems. Other biometrics i.e., iris or Fingerprints recognition is not trustworthy in such situations whereas FR is considered as a fine compromise. Occlusion refers to facade of the face image which can be due to hair, moustache, sunglasses, or wrapping of facial image by scarf or other accessories. Efforts on FR appears in controlled settings have been in the picture for past several years; however identification under uncontrolled condition like partial occlusion is quite a matter of concern. Based on review of literature, a classification is made in this review paper to solve the recognition of face in the presence of partial occlusion. These methods are named as INPAINTING based methods that make use of Diffusion-based Inpainting, Examplar-based Inpainting, Block-matching algorithms, Principal component analysis(PCA),etc

The proposed approach in this paper describes the removal of Occlusion from images using Examplar-based Image Inpainting technique which provides better results and accuracy rather Diffusion-based Image Inpainting technique.

Keywords Occlusion, Removal of Occlusion, Image Inpainting, Diffusion-based Image Inpainting, Examplar-based Inpainting technique.

1. INTRODUCTION

   The Digital Image Processing (DIP) is the analysis and manipulation of digitized image, especially in order to improve its quality and parameters. Almost all of the technical fields are impacted by DIP. It is widely used in Medical Field, Video processing, robot vision, Remote Sensing, etc. Recently the research areas in Images processing recently acquired a lot of attention in analysis of biometrics such as in Iris, Fingerprint, Hand, Ears, Face- Recognition.

   Though tracking and recognizing face objects is a routine task for humans building such a system is still an active research. Among several proposed face recognition schemes, image based approaches are really the most promising ones. However, the images patterns of face objects can dramatically change due to occlusion, lighting and viewing variations. The major portion due to which conflict occurs while Face recognition, is facial occlusion. The research for removal of such facial occlusion is currently an active field where Diffusion based PDA inpainting technique and Examplar based inpainting provides suitable application for disocclusion.[1]

   This paper is organized as follows: Section II comprises literature review; section III comprises Occlusion Overview; section IV describes Inpainting and its different major techniques; section V states proposal of project and Conclusion is in section VI.

2. LITERATURE REVIEW

   Christine Guillemot and Olivier Le Meur [1] presented two major approaches in IMAGE INPAINTING – Overview and recent advances. They proved that system comprise of Diffusion-based inpainting or Examplar- based inpainting techniques have major application as image disocclusion or Removal of Occlusion from Images and such systems may be discovered in order to improve results and accuracy. Mahroosh Banday and Richa Sharma[2], presented a paper in which A comparative study to provide a comprehensive visualization of different types of image inpainting techniques were discussed. A brief description along with advantages and drawbacks of several Inpainting techniques such as diffusion based, texture synthesis, examplar based, hybrid inpainting technique etc are presented properly. In dec-2012, Rui Min and Jean-Luc Dugelay in [3] presented the system where solution to newly identified facial occlusion problem viz. sparse occlusion in the context of Face Biometrics in Video surveillance. Experiments demonstrate that the proposed approach significantly improve various face recognition algorithms in presence of complex sparse occlusions. In their system, they used Fields-of-Experts priors via spatial influence along with PCA framework which required to remove occlusion based on non-occluded part in image.

   Marcelo Bertalmio and Guillermo Sapiro,[4] experimented a lot and proposed a robust method of Image Inpainting in which they introduced a novel algorithm for digital inpainting of still images that attempts to replicate the basic techniques used by professional restorators. After the user selects the regions to be restored, the algorithm automatically fills in these regions with information surrounding them. Their experimental results stated that The inpainting algorithm presented has clearly motivated by and has borrowed from the intensive work on the use of Partial Differential Equations (PDEs) in image processing and computer vision. Aleix M. MartõÂnez, Member of IEEE in june-2002 described in [5] a system to make the recognition system less sensitive to the differences between the facial expression displayed on the training and the testing images,

   they weight the results obtained on each local area on the basis of how much of this local area is affected by the expression displayed on the current test image. In that paper, they proposes several approaches which deals with some of the difficulties that one encounters when trying to recognize frontal faces in unconstrained domains and when only one sample per class is available to the learning system domain.

3. OCCLUSION OVERVIEW

   OCCLUSION is defined as an Obstacle or any unnecessary-object in image disturbing the matching/recognizing process and Occlusion in an image refers to hindrance in the view or area of an object. Possibly it can be natural as well as synthetic. Natural occlusion refers to obstruction of views between the two image objects without any intension while synthetic occlusion refers to artificial blockade of intentionally covering the images view with a white/black solid rectangular block. Sometimes ears are occluded using earings or scarfs, eyes are occluded using eye-lashes. Partial occlusion has been found in many areas and regions of image processing. Even in real time application face image becomes occluded via accessories such as

   Figure 1:- Kinds of occlusion

   • Sunglasses, scarf, beards, hat.

   • Hand on face

   • Face dirt

   • Face behind Fence

   • Texture on Face images, etc.

   Problems Arises Due To Occlusion

   Consider you are developing a system which tracks objects (people, cars, etc.) then occlusion occurs if an object you are tracking is hidden (occluded) by another object. If you are using a range camera, then occlusion are areas where you do not have any kind of information. Facial occlusion is a critical issue in many Face recognition system such as in video surveillance. Many problems occur such as when images are often corrupted by noise, scanned old photo paper, dust resting on the scanning glass of a scanner, scratched images or others have logos or stamps.

4. IMAGE INPAINTING

   Image In-painting, the technique that aims to revert deterioration (scratches, artifacts in photographs and videos) in images in an undetectable form, is as ancient as artistic creation itself. Digital Image In painting, a relatively young research area is an art of filling in the missin or corrupted regions in an image using information from the neighbouring pixels in a visually plausible manner, while restoring its unity. In painting which is essentially an image interpolation problem has numerous applications. It is helpfully used for object removal in digital photographs, image reconstruction, text removal, video restoration, special effects in movies disocclusion and so on. Several approaches have been proposed by the researchers to correct the occlusion where two major approaches used widely are diffusion-based inpainting and Examplar-based Impainting.[1]

   1. Diffusion Based Inpainting:-

      The first category of methods, known as diffusion-based inpainting, introduces smoothness priors via parametric models or partial differential equations (PDEs) to propagate (or diffuse) local structures from the exterior to the interior of the hole (as shown in Figure 2, where U denotes the unknown part or the hole to be filled in, and S the source or known part of the image). Many variants exists using different models (linear, nonlinear, isotropic, or anisotropic) to favour the propagation in particular directions or to take into account the curvature of the structure present in a local neighbourhood. These methods are naturally well suited for completing straight lines, curves, and for inpainting small regions. They, in general, avoid having unconnected edges that are perceptually annoying. However, they are not well suited for recovering the texture of large areas, which they tend to blur. Thus, this method is not compatible.

   2. Examplar Based Inpainting:-

   The second category of methods is based on the seminal work of Efros and Leung[6] and exploits image statistical and self-similarity priors. The statistics of image textures are assumed to be stationary (in the case of random textures) or homogeneous (in the case of regular patterns). The texture to be synthesized is learned from similar regions in a texture sample or from the known part of the image. Learning is done by sampling, and by copying or stitching together patches (called examplar) taken from the known part of the image. The corresponding methods are known as examplar- based techniques. Examplar-based inpainting synthesize entire patches by learning from patches in the known part of the image and these methods are faster than PDE pixel based approaches.

   Some variants have been introduced to optimize these methods:-

   • Distance matrics (SSD).

   • BEST MATCHES using KNN Algorithm.

   • Patch Processing Order.

   • Block Matching Algorithm.

   • Patch Stitching with Blending and Quilting.

5. PROPOSED SYSTEM

   The proposed system uses Examplar-Based Image inpainting technique in order to remove occlusion form images. Diffusion based inpainting is the another technique for image inpainting but having some limitations such as output images may consist of Blurred regions, edges would not appear consisely, failed if texture regions are large,etc. Examplar-based inpainting technique reduces such limitations to certain extent.

   The flow of proposed system is as follows:-

   Figure 2:- Proposed system architecture.

   Input Image:- Input image is a photograph, natural image and synthetic image for processing that contain coloured images of different size etc. that is given to image inpainting.

   Mask Image:- It Removes the unwanted object either single or multiple if required from the original image through paint called mask image or target which is to be inpainted.

   Incomplete Salient Structure Completion:- Connecting incomplete salient structures is the key to obtain creditable inpainting results. However, simple connection or extension of the incomplete salient structures results texture features are used to determine the similarity of incomplete salient structures.

   Texture Propagation of Completed Salient Structures:- After structure completion, we propagate texture information into the target region through patch based processing order variant of Examplar-based inpainting technique.

   Inpainted output Image:- The output will be required image which is completely inpainted image where it appears as occlusion is removed from the occluded input image.

6. CONCLUSION

Merging Examplar based Image Inpainting technique with Block matching algorithm in a single unit, Occlusions will be removed from images and An efficient system may be developed with better accuracy to achieve required results. Existing Face recognition system may use such systems as well, as a preprocessing unit to enhance & improve accuracy in results.

REFERENCES

1. Christine Guillemot and Olivier Le Meur, Image Inpainting IEEE signal processing magazine, JANUARY 2014, 1053-5888/14

2. Mahroosh Banday and Richa Sharma, Image Inpainting- An Inclusive Review of the Underlying Algorithm and Comparative Study of the Associated Technique, International Journal of Computer Application (0975-8887) vol no 98, no 17, july 2014.

3. Rui Min and Jean-Luc Dugelay, INPAINTING OF SPARSE OCCLUSION IN FACE RECOGNITION IEEE transactions Feb. 2012.

4. M. Bertalmio, G. Sapiro, C. Ballester, and V. Caselles, Image inpainting, in Proc. ACM SIGGRAPH, July 2000, pp. 417424.

5. Aleix M. MartõÂnez,, Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 24, NO. 6, JUNE 2002.

6. A. Efros and T. Leung, Texture synthesis by non-parametric sampling, in Proc. Int. Conf. Computer Vision(ICCV), Sept. 1999, pp. 1033-1038.

7. Xiaoyang Tan, Songcan Chen, Zhi-Hua Zhou, Face Recognition under Occlusions and Variant Expressions with Partial Similarity. IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, AUG 2008.

8. Irene Kotsia, Ioan Buciu, Ioannis Pitas, An analysis of facial expression recognition under partial facial image occlusion SCIENCE DIRECT Image and Vision Computing 26 (2008) 10521067

9. O. Le Meur, J. Gautier, and C. Guillemot, Examplar-based inpainting based onlocal geometry, in Proc. IEEE Int. Conf. Image Processing (ICIP), Sept. 2011, pp.34013404.

10. J. Sun, L. Yuan, J. Jia, and H. Shum, Image completion with structure propagation, ACM Trans. Graph., vol. 24, no. 3, pp. 861 868, 2005.

11. J. Aujol, S. Ladjal, and S. Masnou, Examplar-based inpainting from a variational point of view, SIAM J. Math. Anal., vol. 42, no. 3, pp. 12461285, Mar. 2010.

12. A. Criminisi, P. Pérez, and K. Toyama, Region filling and object removal by examplar-based inpainting, IEEE Trans. Image Processing, vol. 13, no. 9, pp. 12001212, Sept. 2004.