An Efficient Image Retargeting By Enhanced Face Saliency Technique

An Efficient Image Retargeting By Enhanced Face Saliency Technique

An Efficient Image Retargeting By Enhanced Face Saliency Technique

Asst.Prof.Beena.M.V

Department of Computer Science and Engineering Vidya Academy Of Science & Technology Thrissur, India

Niya Prasad

Dept. of Computer Science & Engineering Vidya Academy Of Science & Technology Thrissur, India

AbstractAdvancement of multimedia technology leads to the significant requirement in convenient data access using portable devices, such as mobile phone and PDA, for transmitting, sharing and exchanging information. The existing digital images, such as those taken by digital cameras, usually have high resolution and fixed aspect ratio. They should be adapted for effective viewing on a screen with small size and arbitrary aspect ratio. This adaptation is commonly referred to as image retargeting. The identification of relevant objects in an image is highly relevant in the context of image retargeting. Especially faces draw the attention of viewers. But the level of relevance may change between different faces depending on the size, the location, or whether a face is in focus or not. The paper presents method for image retargeting ,mainly considering human faces. The overall process for retargeting is divided in to three steps. The method first find the faces from the input image and then finding the focused face.After that it is retargeted to the target size.

1. INTRODUCTION

   With the recent advances in imaging technology, digital images have become an important component of media distri- bution. Images are frequently used in news stories, and people post their pictures on-line .

   Since Mobile phones and personal digital assistants (PDAs) typically have limited resolution due to their small form factor. Images, however, are typically authored once, but need to be adapted for consumption under varied conditions. Image retargeting (IR) is the process of mapping the original image to the target zone. Mobile devices are widely used for acquiring, displaying and editing visual media such as images and videos, its becoming more and more common that adapting the sizes of these media so that they can be shown on different devices. As all kinds of digital imaging devices have been invented and widely used, users can easily shoot as many photos as they can. However, most daily users are lack of aesthetic experience, therefore its a difficult task to touch up and improve these photos automatically.

2. IMAGE RETARGETING PROBLEM

   A digital image of size m n can be represented by a 2D discrete grid of pixels with m rows and n columns,where each

   Fig. 1. An example of image retargeting: The source image containing three areas of importance, the two boys, and the ball. center(left). The source image retargeted to PDA display and cellphone display( right)

   pixel has a value that encodes its color or intensity information. The image retargeting problem can be defined as . Given an image I of size mn and a new size m n, the goal is to produce a new image I of size m n that will be a good representative of the image I.

   The three main objectives for retargeting are:

   • The important content of I should be preserved in I.

   • The important structure of I should be preserved in I.

   • I should be free of visual artifacts.

   This is a very challenging problem due to a number of reasons. Real-world scenes exhibit tremendous variability, and the techniques are expected to handle different kinds of imagery, such as barren landscapes and complex scenes with no clear foreground-background segmentation. Images taken outdoors have different overall characteristics from images taken indoors, and the presence of elements such as faces and text also may bring different meanings to specific image regions. Crowded and complex scenes are typical failure cases of current algorithms, due to the issues on automatically determining what is considered important in the image.

   The previous works on the image retargeting mainly focused on the other images not of image containing faces and also the methods like scaling,cropping does not provide efficient method for retargeting because it produces visual distortion to the images.In this paper,first find the faces from the input image and then finding the focused face.After that it is retargeted to the target size.

   The rest of this paper is organized as follows. In Sec. II and III, Presents the Image retargeting steps: Face Detection,Focus Detection and the Retargeting Method. In Sec. IV, results and discussions. Finally, Sec. V concludes this

   paper.

   Fig. 2. Steps involved in the retargeting of an image

3. IMAGE RETARGETING METHOD

   Principally,image retargeting can be defined as the prob- lem of adapting an image to various target screens. It is a technique that adjust input images in to arbitrary sizes and simultaneously preserve the salient regions of the input images[6]. The identification of relevant objects in an image is highly relevant in the context of image retargeting(IR). Especially faces draw the attention of viewers. So an efficient retargeting considering the human faces is essential[1].

   The three main steps for the retargeting of an image used here are:

   1. Face Detection

   2. Focus detection

   3. Retargeting

   The faces are detected from the input image by using a face detector and from the detected faces,focused face is found using image gradient function. This method is used to exclude all blurry faces. Each face includes characteristic edges due to nose, mouth, eyes and eyebrows, among others. A face is classified as in focus when at least one of its edges is classified as a strong edges,i.e faces with strong edges are considered as focused face. Once the focused face is detected, retargeting can be performed by considering the energy of pixels.

   The overall process can be easily explained by the following fig2 & 3.

   1. FACE DETECTION

      In the face detection (FD) part, faces are de- tected from the input image by using function vi- sion.CascadeObjectDetector[20]. It detects the location of a face in a input image. The cascade object detector uses the Viola-Jones detection algorithm and a trained classification model for detection. By default, the detector is configured to detect faces.Here I have given input image (jpg) that contain human faces is given to the system. Here face De- tector = vision.CascadeObjectDetector[20]. By using the vi- sion.CascadeObjectDetector the faces are detected.The image with detected faces is obtained after processing as output.

      CascadeObjectDetector Detect objects using the Viola-Jones algorithm. DETECTOR = vision.CascadeObjectDetector creates a System object

      Fig. 3. overall Design

      that detects objects using the Viola-Jones algorithm. The DETECTOR is capable of detecting a variety of objects, including faces and a persons upper body. The type of object to detect is controlled by the Classification Model property. By default, the DETECTOR is configured to detect faces[20]. BBOXES = step(DETECTOR,I) performs multi-

      scale object detection on the input image, I, and returns, BBOXES, an M-by-4 matrix defining M bounding boxes containing the detected objects. Each row in BBOXES is a four-element vector, [x y width height], that specifies the upper left corner and size of a bounding box in pixels. When no objects are detected, BBOXES is empty. It must be a gray scale or true color (RGB) image.

      By using the funtion RGB = insertObjectAnnotation(I, SHAPE, POSITION, LABEL),inserts rectangles and corre- sponding labels at the location indicated by the position matrix. The POSITION must be an M-by-4 matrix, where each row, M, specifies a rectangle as a four-element vector, [x y width height]. The elements, x and y indicate the upper- left corner of the rectangle, and the width and height specify the size.

   2. FOCUS DETECTION

   The image containing detected faces is used as input in the focused detection part.Here the input includes all the detected faces in an image. Mainly this method is used to exclude all blurry faces. Each face includes characteristic edges due to nose, mouth, eyes and eyebrows, among others.

   A face is classified as in focus when at least one of its edges is classified as a strong edge.

   In the focus detection part ,the maximum in- formation contained face from the detected faces is found. This can be done by finding weight/density/energy of pixel. For that Image gradient function is used. The imgradi- ent(I)=[Gmag,Gdir] returns the gradient magnitude and gra- dient direction for the image I. The input image I can be gray scale or binary image and data type can be single,double, int8,int32,unit8,unit16,unit32,logical. The method is used by

   gradient operator,(go) specified as one of the text strings Sobel,Prewitt, Roberts.Directional gradients along x-axis and y-axis specified as non-sparse matrices equal in size to image I, they are obtained from function imgradientxy. The output arguments Gmag returned as non-sparse matrix the same size as image I and also Gdir,non sparse matrix the same size as image I. By the gradient function,energy/edge visibility can be found out. The energy of a pixel measures how much that pixel stands out from its surroundings and give a rough idea about the importance of that pixel. The energy can be calculated from gradient image, by using an equation.

   So by energy measure, the focused face is detected,i.e those faces are having high energy than the other smoothened face.

4. IMAGE RETARGETING STEPS

   1. RETARGETING METHOD

      Retargeting is a continuous process after the focus de- tection, it is essential to check that whether there exist faces that have almost same or appropriate energy. If there are two faces two faces that have appropriate energy then those two faces must be considered. If two faces are in different position then the area in which the two faces exist is considered. So the area in which the two faces which has appropriate energy is selected. Then add 10 percent to the selected area. Then after that for creating a rectangular box for that selected area, again add 40 percent to the selected area by adding pixels left,right,up and down of the area. So this selected area is considered as focused area.

      The next step is to check that there exist any energy in the background.For checking that, the total energy of the image is need to be found. Then checking that 80 percent of the total energy of the image is available in the focused area. If it is not available in the focused area then energy is still existing in the background.So in that case,again adding extra 10 percent to the focused area then check that the condition is satisfied or not.If not satisfied then again add extra 10 percent to the focused area. This process continues until the condition is satisfied. Once it is satisfied then stop adding and by cropping that area the final retargeted image is obtained. The output obtained is retargeted image.

   2. Improved Face Detection Method

   Face detection is an important part in this project and efficient image retargeting requires a system that detects all the faces in the input image. In the method described above, I used face detector, by default it automatically detect the faces in an input image. But during the detection process in some cases the detector is not able to detect all the faces in the input image. If the face detection is not properly done, it will seriously effect the focus detection part and also the retargeting. So a better method for face detection is needed to be considered.

   This improved face detection method is based on the neural networks. The method provides two template i.e, template1 and template 2, the two template are two faces

   Fig. 4. Architecture of Improved Face Detection

   Fig. 5. Yellow dots shows the comparing input with template faces

   which is of size 18×27 width and height and binary image. The first step of the detection part is that system matches the area of the two templates with the input image. This is done by convolution and correlation method.The template1 and template 2, the two template are two faces which is of size 18×27 width and height and binary image.The first step of the detection part is that system compares the input image with two templates. (fig 5). This is done by convolution and correlation method. The input given is a png image with size 150×150. So the first step is about comparing with the template faces. Then in the next step comparing with the faces in the database(fig6). If it found match, then it consider as face, if not found match then no face will be detected(fig7). In such cases training is done to the system, and again checking to found match. The output of the detection gives image with detected faces.

   Fig. 6. pink dots shows the comparing input with database contain faces

   Fig. 7. By finding match, face is detected

5. RESULTS AND DISCUSSIONS

   This chapter discuss about the output obtained after performing the retargeting of an image described in the previous chapter. To the best of my knowledge ,a standard face image data set is currently available for benchmarking. Existing image retargeting were not done much on image con- taining faces. Also the automatic face detector was available was used for detection of faces from input images.

6. FACE DETECTION RESULTS

   The vision.CascadeObjectDetector was used to detect the location of a face in a input image. The cascade object detector uses the Viola-Jones detection algorithm and a trained classification model for detection. By default, the detector is configured to detect faces. So the automatically detected faces is then used to find the focused face. The result of the face detection part is shown below in fig 8.

   Fig. 8. Result

7. FOCUS FACE DETECTION AND RETARGETING RESULTS

   In this method the focused face is found from the detected faces. A focus detection algorithm is used to exclude all blurry faces. The approach to the edge detection problem is to analyze luminance variations in an image. We use the gradient magnitude to identify edge pixels. So in the focus detection part find the maximum information contained face from the detected faces. The result of the focus detection and the retargeted image is shown in fig 8. Also some of the examples of the retargeted image is shown in fig 13 fig 14 and fig 15.

8. IMPROVED FACE DETECTION RESULTS

   Face detection in the above method was not able to detect faces to some inputs. Since face detection is one of the important part, an enhanced or better approach is used which can detect all the faces of the input image in which other failed (fig 9 & fig 11). So by comparing the two method this method seems to be better for the face detection. This is shown in fig 10 & fig 12.

9. CONCLUSION

In this paper, a method for finding the faces that are in focus is presented, since this paper mainly concentrate on images that contain faces. This information is especially useful in the context of image retargeting, where detections of unimportant faces in the background may lead to visual distortions of really important faces. Faces are first detected with the use of face detector which is configured to detect faces automatically. As faces that contain maximum energy are assumed to be in focus, and used the gradient magnitude to classify the fces to be in or out of focus. A retargeting method is finally used to retarget the image that contain focused face. Also another improved face detection method is used for better detection of faces.

Fig. 9. Output of the Face Detection Result1

Fig. 10. Output of Improved Face Detection Result1

Fig. 11. Output of the Face Detection Result2

Fig. 12. Output of Improved Face Detection Result2

Fig. 13. Some Examples of the Image Retargeting Method

Fig. 14. Some Examples of the Image Retargeting Method

Fig. 15. Some Examples of the Image Retargeting Method

REFERENCES

1. Johannes Kiess, Rodrigo Garcia, Stephan Kopf, Improved image re- targeting by distinguishing between faces in focus and out of focus . ICME Workshops, page 145-150. IEEE, (2012).

2. Chen, B., and Sen, P. 2008. Video carving. In Short Papers Proceedings of Eurographics.

3. Chen, L., Xie, X., Fan, X., Ma, W., Zhang, H., and Zhou, H. 2003. A visual attention model for adapting images on small displays. Multimedia Systems 9, 4, 353364.

4. Gal, R., Sorkine, O., and Cohen-or, D. 2006. Feature aware texturing. In Eurographics Symposium on Rendering, 297303.

5. Guo, Liu, F., Shi, J., Zhou, Z.-H., and Gleicher, M., Image retargeting using mesh parametrization, IEEE Trans. on Multimedia 11 (5), 856867 (2009).

6. Achanta and S usstrunk, S., Saliency Detection for Content-aware Image Resizing, in [ IEEE Intl. Conf. on Image Processing ], (2009)

7. Keysers, D., and Unger, W. 2003. Elastic image matching is np- complete. Pattern Recogn. Lett. 24, 1-3, 445453.

8. Liu, F., and Gleicher, M. 2006. Video retargeting: automating pan and scan. In Proc. of the 14th annual ACM international conf. on Multimedia, ACM, 241250.

9. Liu, H., Xie, X., Ma, W., and Zhang, H. 2003. Automatic browsing of large pictures on mobile devices. Proceedings of the 11tn ACM international conf. on Multimedia, 148155.

10. Liqun Chen, Xing Xie, Xin Fan, Wei-Ying Ma, Hong-Jiang Zhang, and Heqin Zhou, A visual attention model for adapting images on small displays, ACM Transactions on Multimedia Systems, vol. 9, pp. 353364, November 2003.

11. Michael Rubinstein, Ariel Shamir, and Shai Avidan, Multi operator media retargeting, ACM Transactions on Graphics, SIGGRAPH 2009, vol. 28, no. 3, pp. 111, 2009.

12. Michael Rubinstein, Diego Gutierrez, Olga Sorkine, and Ariel Shamir, A comparative study of image retargeting, in ACM SIGGRAPH Asia 2010 papers, New York, NY, USA, 2010, pp. 160:1160:10, ACM.

13. Radhakrishna Achanta, Sheila Hemami, Francisco Estrada, and Sabine Susstrunk, Frequency-tuned salient region detection, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15971604, June 2009.

14. Radhakrishna Achanta, Francisco Estrada, Patricia Wils, and Sabine Susstrunk, Salient region detection and segmentation, International Con- ference on Computer Vision Systems (ICVS), vol. 5008, pp. 6675, 2008.

15. R. Osadchy, M. Miller, and Y. LeCun, Synergistic Face Detection and Pose Estimation with Energy-Based Models, J. Machine Learning Research, vol. 8, pp. 1197-1215, 2007.

16. Santella, A., Agrawala, M., Decarlo, D., Salesin, D., and Cohen, M. 2006. Gaze-based interaction for semiautomatic photo cropping. In ACM Human Factors in Computing Systems (CHI), 771780.

17. Setlur, V., TakagiI, S., Raskar, R., Gleicher, M., and Gooch, B. 2005. Automatic image retargeting. In In the Mobile and Ubiquitous Mul- timedia (MUM), ACM Press.

18. Shai Avidan and Ariel Shamir, Seam carving for content aware image resizing, ACM Transactions on Graphics, SIGGRAPH 2007, vol. 26, no. 3, 2007.

19. Setlur, Takagi, S., Raskar, R., Gleicher, M., and Gooch, B., Automatic image retargeting, in [ ACM Intl. Conf. on Mobile and Ubiquitous Multimedia ], 5968 (2005).

20. Viola, P. and Jones, M., Rapid object detection using a boosted cascade of simple features, in [ IEEE Conf. on Computer Vision and Pattern Recognition ], (2001).

21. V. Balasubramanian, J. Ye, and S. Panchanathan, Biased Manifold Em- bedding: A Framework for Person-Independent Head Pose Estimation,

    Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007

22. Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkin, and Tong-Yee Lee, Optimized scale-and-stretch for image resizing, ACM Transactions on Graphics (Also Proceedings of ACM SIGGRAPH ASIA), vol. 27, no. 5, 2008.

23. Y. Li, S. Gong, J. Sherrah, and H. Liddell, Support Vector Machine Based Multi-View Face Detection and Recognition, Image and Vision Computing, vol. 22, no. 5, p. 2004, 2004.