Morphing Techniques for Facial Images – A Review

Morphing Techniques for Facial Images – A Review

Shubhangi V. Lokhande

M.E. Scholar

D.Y. Patil College of Engineering & Technology, Kolhapur

S. B. Patil

Professor

D.Y. Patil College of Engineering & Technology, Kolhapur

Abstract

Image metamorphosis, commonly known as morphing, is used to generate a sequence of images that transform a source image into a target image. This technique is often used to create special effects for motion pictures or television. Image metamorphosis is performed by coupling image warping with color interpolation. The idea is distorting the first image into the second one and vice versa. This paper discusses an overview and challenges of different morphing techniques for manipulating two-dimensional human facial images.

Keywords- Face Morphing, cross-dissolve, warping, pixels.

1. Introduction

   Face morphing technique is very widely used in various fields like computer animations, games and movies. The main concept is to create an intermediate image by mixing the pixel color of the original image with another image. In this method a source face image is transformed into a destination face image smoothly by interpolating some in-between images; hence, the interpolated images contain similar features to both the source image and the destination image. For such interpolation between two face images, the corresponding pixels between the input images must be specified. Once both images have been warped into alignment for intermediate feature positions, cross- dissolve generates in between images.

   Here, warping is the distortion in an image according to a mapping between a source space(u,v) and a target space(x,y)[1,2]. Image warping is used in image processing primarily for correction of geometric distortions. Cross-dissolve is a colour transformation

   ( i.e, a group of transformations of the image colour space). In general, image morphing is the combination of image warping with a cross-dissolve between images. As the morphing proceeds, the first images of the sequence resemble the source image, while the last images are similar to the target image. The middle

   image contains an average of the source and the target image adjusted to average feature geometry.

2. Problem Definition

   The feature correspondence is used to compute the mapping functions that define the spatial relationship between all points in both images. Feature specification is the most tedious aspect of morphing. Although the choice of allowable primitives may vary, all morphing approaches require careful attention to the precise placement of primitives. Given feature correspondence constraints between both images, a warp function over the whole image plane must be derived. This process, referred to as warp generation, is essentially an interpolation problem. Another interesting problem in image morphing is transition control. If transition rates are allowed to vary locally across middle images, more interesting animations are possible. [3]

3. Literature Review

   There are number of advantages in applying face morphing technique in various kinds of work. We can categorize them into five groups, based on their corresponding pixel mapping method.

   1. Cross-Dissolving Method

      The easiest way of morphing is cross dissolving, because this method uses the pixels located in the same position of the source image and the destination image to calculate the result. In this method there is overlapping of colors and positions in the result image transition. [4, 5, 6]

      Fig.1 Cross-Dissolve

   2. Mesh Warping

      This method uses a non-uniform mesh to define control points between the source image and the destination image. The mesh table of the source image is mapped to another mesh table on the destination image. The calculation is taken on each mesh, which should be easily computed. However, to use mesh warping methods, the source image and destination image should be rather similar in shape because a pixel will be mapped to another pixel with the positions on the mesh tables [3, 4, 6, 7, 8].

   3. Field Morphing Method

      This is a method that uses lines to mark the main features of the face such as the eyes, nose, mouth and edges of the face. The computation uses the lines of the source image, which are mapped to the same type of line in the destination image. For example, the line that marks the nose in the source image should be mapped to the line that also marks the nose in the destination image as well. [6, 7, 8, 9, 10]

   4. Point Distribution Method

      This method uses points that the users fix to each main feature of the face, to help map the source image and the destination image together. The computation uses these points to calculate the result images. The resulting images from this work are satisfactory to the users. However, it is not automatic, because the users have to fix the mapping point of the features before making the program merge them together. [11, 6]

   5. Critical Point Filters Method

   The critical point filters method can extract the main features of the face by using the color difference in the features. The maximum sub-image can extract the eye and hair of the face, the max-min saddle sub- image can extract lips, the min-max saddle sub-image can extract the skin and the minimum sub-image can extract the background of the image [6,12,13].

4. Morphing Techniques

There are a variety of morphing methods in the literature [14]. Each of these methods has its own advantages and disadvantages. There are two techniques that are widely used to warp images: mesh warping field morphing and. field morphing will be briefly addressed in a later section.

1. Triangle Mesh Warping:

   Triangle mesh warping consists of non-uniform grids having a suitable set of triangles with the given data points being the corners of the triangles (called triangulation). The source image is referred to as IS and the target image is referred to as IT. The source image has an associated mesh MS that specifies the coordinates of different features. A second mesh, MT, specifies their corresponding positions in the target image. Facial features such as the eyes, nose, and lips should lie below corresponding grid lines in both meshes. Together, MS and MT are used to define the spatial transformation that maps all points in IS onto IT. In the meshes no folding or discontinuities are permitted (i.e., topologically equivalent). Therefore, the nodes in MT may wander as far from MS as necessary, as long as they do not cause self-intersection.[10] Catmull-Rom cubic spline is connected to the mesh points. Figure 2 shows warping a scan-line in 3×3 meshes.[8]

   Fig.2. Warping a scan-line

   There is a mapping function between the scan-lines using the intersection points. Use Catmull-Rom cubic spline to interpolate and map each pixel on the scan- line of the auxiliary image to the pixel on the scan-line in the original image. We need to find the coefficients of the Catmull-Rom cubic spline and substitute x to find the mapping. Curve equation is

   y(x) = ax3 + bx2 + cx + d y(x) = 3ax3 + 2bx2 + c

   from the properties of Catmull-Rom spline, y(0) = d = y2

   y(0) = c = (y3 y1) / (x3 x1) r = x3 x2

   y(r) = ar3 + br2 + cr + d = y3

   y(r) = 3ar2 + 2br + c = (y4 y2) / (x4 x2)

   This method use two-pass algorithm means warping in horizontal direction followed by warping in vertical direction. It performs 2D image warping computation using a combnation of two 1D warping computation.

   1. Triangulation

      Several different triangulation techniques can be used for triangulation of a set of points. Triangulation of a set of points is a process that is done in computer graphics [15]. Delaunay triangulation [16] is a popular method for optimal triangulation of a set of points. To calculate the Delaunay triangulation, Voronoi regions [16] should be obtained. Many different direct and indirect construction methods are available for the Delaunay triangulation. The method that is adopted in this work is the combination of the quality triangular mesh generation proposed by Ruppert

      [17] and the divide-and conquer incremental Delaunay triangulation proposed by Guibas and Stolfi [16]. These two triangulation techniques have been reported to generate more accurate results than the existing counterparts and to be faster than them. A triangle is coded by its address (position) in the graph. Voronoi diagram and Delaunay triangulation of a sample face image is shown in Figure 3. A set of 80 distinct points are used for computation of the Delamay triangulation.

      Figure 3: Triangulation of a face image using 80 distinct points. a) Voronoi diagram. b) Delaunay triangulation.

2. Field Morphing:

   Another alternative morphing technique is to warp images with a field morphing Algorithm. Field morphing was introduced by Beier and Neely (1992).[9]

   The user has to provide the algorithm with a set of control line segments. The segments serve to align the features of images A and B. The more control line segments there are, the more control the user has over these shape changes. For faces, for example, the line segments can be drawn over the contour line segments of the head, the ears, the eyes, the eyebrows, the lips, the nose, and so on.

   Fig.4 Field Morphing

   In Figure 4,127 segments are drawn over the contours and features of the face. In the warping phase of field morphing, correspondences are calculated between the pixels of the images to be morphed. The algorithm is called field morphing because every line segment exerts a field of influence on the alignment such that pixels near a segment in one image will tend to be aligned with pixels near the corresponding segment in other images. Under a certain parameter setting, the algorithm guarantees that pixels on a segment in one image will be aligned with the pixels on the corresponding segment in the other images.

   I will now first explain the field morphing method with just a single control line segment and then explain the method with multiple control line segments.

   Fig.5 (a) Single Control Line Segment (b) Two control line segments

   1. Single Control line Segment

      Let us first discuss the case in which there is only a single control line segment in two images. Let i = 0 to refer to the destination image, and i = 1 and i = 2 refer to the two source images, respectively. The control line segments are defined in terms of a pair of vectors.

      n: length which goes from the origin of the coordinate system to the starting location of and the other representing its orientation)

      m: length which goes from the starting location to the ending location of segment li

      Now the first step in field morphing is to calculate the

      coordinates for the line segment in the destination image on the basis of the coordinates of the line segments from the source images. The following equations transform the segments l1 and l2 in the source images into the segment l0 in the destination image by transforming m1 and m2 into m0 and by transforming n1 and n2 into n0:

      m0 = m1 + (m2 m1) n0 = n1 + (n2 n1)

      Where, : relative influence of the segments of the first and the second image (varies between 0 and 1), although values outside this range will be used for generating caricatures.

   2. Multiple control line segments:

      Figure 5a shows warping with two line segments. Each image now has two control line segments. For every position vector P0, calculations for each segment in each of the two source images lead to possibly different position vectors. A unique position vector Pi is calculated for image i by combining all individual position vectors in a weighted average. Various labels are omitted for reasons of clarity. Beier and Neely's algorithm [9] weights the contributions of each segment to obtain a single source coordinate pi. Weight of each line pair depends on the length and distance.

      pij : source coordinate, where j indexes the control line segment.

      dij : a displacement vector, which is a displacement from po to pij.

      The individual displacements dij are summed in a weighted average to obtain the single displacement toward pi. Each weight is a function of the distance from p0 to the segment j and the length of segment j. The weight functions used are:

      Where a, b and c are parameters. The second step is the calculation of the distortions of the two images toward the new line segments. There are two ways of doing this: a forward mapping and a reverse mapping.[8]

   3. The advantage of the field morphing method over the mesh warping method:

In field morphing there is ease of aligning the line segment to features present in an image. In the mesh warping method, when diagonal contours are present in the image, one can pick either an initially vertical or an initially horizontal edge of the grid and align it to part of the diagonal contour. It can be difficult to make these choices when the image contains contours that change orientation (such as the contour of a face).

It takes more time to compute a morph than with the mesh warping method.

1. Conclusion:

   Morphing algorithms commonly share some components such as feature specification, warp generation, and transition control. The ease with which researchers can effectively use morphing tools is determined by the manner in which these components are addressed. We briefly surveyed widely used morphing techniques such as mesh warping and field morphing.

2. References:

1. PS Heckbert, Fundamental of texture mapping and image warping, M.S. thesis, Dept. of Electrical and Electronics Engineering, University of California, Berkeley, CA 94720,June 1989.

2. G Wolberg, Digital Image Warping, IEEE Computer Society Press, Los Alamitos, CA, 1990.

3. AZ Kouzani, S Nahavandi, N Kouzani, LX Kong, FH She. A morphing technique for facial image representation, IEEE Int. Conf. Syst., (2000) Man Cybern., (2), 13781383.

4. T Terada, T Fukui, T Igarashi, K Nakao, A Kashimoto, Y Wei Chen. Automatic Facial Image Manipulation system and Facial Texture Analysis. International Conference on Natural Computation – ICNC , pp. 8-12, 2009.

5. V Zanella, G Ramirez, H Vargas, LV Rasas. Automatic Morphing of Face Images. International Conference on Adaptive and Natural Computing Algorithms – ICANNGA , pp. 600-608, 2009.

6. Areeyapinan, Jennisa, Pizzanu Kanongchaiyos. "Face morphing using critical point filters." Computer Science and Software Engineering (JCSSE), 2012 International Joint Conference on. IEEE, 2012.

7. U Bhushan, GP Saroha, D Tiwari, An Implementation of Image Morphing Through Mesh Morphing Algorithm, International Journal of Advanced Research in Computer Science and Software Engineering, 2012, 2(7), pp. 74-76.

8. Islam, Md Baharul, Md Tukhrejul Inam, Balaji Kaliyaperumal. "Overview and Challenges of Different Image Morphing Algorithms." International Journal of Advanced Research in Computer Science and Electronics Engineering (IJARCSEE) 2.4 (2013): pp-410.

9. Beier T, Neely S (1992). Feature-based image metamorphosis. Computer Graphics, 26 (2), 35-42.

10. Karam, Hussien, A. Hassanien, M. Nakajima. "Feature- based image metamorphosis optimization algorithm." Virtual Systems and Multimedia, Proceedigs Seventh International Conference on. IEEE, 2001.

11. Zanella, Vittorio, Olac Fuentes. "An approach to automatic morphing of face images in frontal view." MICAI 2004: Advances in Artificial Intelligence. Springer Berlin Heidelberg, 2004. 679-687.

12. B Chambers. Point-And Window-Based Matching in Images Using Critical-Point Filters, Masters Thesis, Department of Electrical Engineering University of Illinoist at Urbana-Champaign, 2004.

13. K Thanasoontornlerk, P Kanongchaiyos, An Image Matching Using Critical-Point Filters and Level Set Analysis, International Conferences in Central Europe on Computer Graphics, Visualization and Computer Vision, 2007.

14. George Wolberg, Recent advances in image morphing, in Proc. Computer Graphics Intl., Pohang, Korea, June 1996, pp. 64-71.

15. JD Foley, AV Dam, SK Feiner, JF Hughes. Computer Graphics : Principles and practice. Addison-Wesley Professionals, 2 edition, 1996.

16. MJ Jones, T Poggio, Model-based matching by linear combinations Of prototypes, Tech. Rep.1583, MIT AI Lab., November 1996.

17. JR Ruppert. A delaunay refinement algorithm

for quality 2-dimensional mesh generation. Journal of Algorithms, 1995, 18(3), pp. 548-585.