Image Super Resolution with Sparse Neighbor Embedding and Hog

Image Super Resolution with Sparse Neighbor Embedding and Hog

Nisa A K

Dept.of Computer Science CUSAT

Kochi,Kerala

Meerakrishna G H Dept.of Computer Science CUSAT

Kochi,Kerala

AbstractA fuzzy clustering based sparse neighbor embedding technique for single image super resolution is proposed. When a low resolution image is given similar patches in the training set are selected from the closest cluster.HR image is constructed with the help of neighbor embedding using the similar training patches.The experimental results show that the proposed method is very flexible and gives good results compared with other methods which use neighbor embedding.

Index Terms Fuzzy clustering, neighbor embedding (NE), SL0 algorithm.

1. INTRODUCTION

   In most electronic imaging applications [1], images with high resolution (HR) are desired and often required. HR means that pixel density within an image is high, and therefore an HR image can offer more details that may be critical in various applications such as medical diagnosis, pattern recognition, remote surveillance etc. The most direct solution to increase spatial resolution is to reduce the pixel size (i.e., increase the number of pixels per unit area) by sensor manufacturing techniques. As the pixel size decreases, however, the amount of light available also decreases. It generates shot noise that degrades the image quality severely. Another approach for enhancing the spatial resolution is to increase the chip size, which leads to an increase in capacitance. Since large capacitance makes it difficult to speed up a charge transfer rate, this approach is not considered effective. One promising approach is to use signal processing techniques to obtain an HR image (or sequence) from observed multiple low-resolution (LR) images or from single low resolution image. Such a resolution enhancement approach has been one of the most active research areas, and it is called Super Resolution (SR) image reconstruction.

   SR approaches can be mainly divided into two types: Multi frame super resolution and Single image super resolution. Multi frame SR is the process of recovering a HR image from multiple LR images of the same scene. Single image SR is the process of generating a HR image from a single LR image with the help of a set of one or more training images. Single image SR is also called example-based SR because the HR details of LR image can be predicted by learning the relationship between LR patches and their corresponding HR patches from examples.

   During these years many single image SR approaches have been proposed. Freeman et al. [6] proposed an example based method which uses Markov network to learn the co- occurrence relationship between the local regions of images

   and underlying scenes. But the algorithm heavily depends on a large training data set. Chang et al. [7] introduced locally linear embedding (LLE) from manifold learning, assuming that two manifolds of the LR image patches and the corresponding HR patches are locally in similar geometries. Taking this assumption, the neighbor-embedding-based (NE) algorithm is proposed to estimate HR image patches by linearly combining the HR counterparts of neighbors. But this method fails to establish the LR-HR feature mapping and fixed neighbourhood size usually ends up in blurring effects. Yang et al. [9] proposed an example- learning-based SR via sparse representation .In this the sparse representation of a low-resolution image patch can be applied with the high- resolution image patch dictionary to generate a high- resolution image patch. Dong et al.presented a unified image restoration framework by integrating adaptive sparse domain selection and adaptive regularization, which performs well on image denoising, deblurring, and SR reconstruction. By considering both the local sparsity and the nonlocal sparsity constraints, they presented a sparse representation model for image restoration.

   Gao et al. [10] introduced sparse neighbor embedding (SpNE) which uses Robust-SL0 algorithm [11] for sparse representation and histograms of oriented gradients (HoG)

   [12] for clustering training data set.In SNE [10] Gaussian function is used for smoothening the l0 norm and K-means clustering is used. It takes much time to converge as the dimensionality increases.

   Proposed method uses sigmoid function for smoothening L0 norm and fuzzy c means clustering for clustering purpose. Method is evaluated quantitatively and subjectively and produces better result compare to other methods.

   The paper is organized as follows. Section II describes our proposed method. Section III provides experimental results.

2. PROPOSED METHOD

   The proposed SR method incorporates the idea of clustering and neighbour embedding in HR reconstruction. The proposed method estimate the HR details of the input LR image with the help of a training set. The training set consists of LR- HR patch pairs. The proposed method constructs the HR image by combining the neighbour (similar) training HR patches. The way in which the HR patches are combined is based on neighbour embedding algorithm. For each input LR patch, neighbour embedding algorithm find the similar LR patches from the training set. Then compute the reconstruction weights of the similar LR patches using sparse

   neighbourhood selection that minimize the reconstruction error i.e., the input LR patch can be reconstructed from the similar training LR patch with least error. These weights are used for constructing the target HR patch. The target HR patch is represented as the weighted sum of training HR patches corresponds to the similar (neighbour) training LR patches. Because of the diverse structures in real images, the similar training patches selected for reconstruction may not be compatible to each other which will affect the visual quality of target image. To make the patches compatible with each other, we have to find the relationship between the training patches. To learn the relationship between training LR patches, proposed method uses fuzzy clustering in the training phase. And this information is further used in the selection procedure of neighbour embedding for selecting compatible training patches. Since the inherent relationship within the training set is also used in the patch selection, the proposed method improves the visual quality of the target HR image compared to other existing approach. An overview of the proposed method is illustrated in the following block diagram.

   Input LR

   Sl0 optimization using sigmoid

   Neighbor Embedding

   Output HR

   Fig.1 Overview of the proposed method

3. RESULTS AND DISCUSSIONS

   In example based super resolution, the selection of training set is very important for the reconstruction quality of HR image. The images are taken from the Kodak web site. To mimic the real imaging system, all the training images are blurred by a 9x 9 Gaussian filter with standard deviation 1.1 and down sampled by a decimation factor of 3 to produce the corresponding LR training images. Since the human visual system is more sensitive to the luminance channel than the chrominance channels, we transform RGB values into YCbCr color space and only carry out the SR process on this part. We directly magnify the chrominance (Cb and Cr) channels to the desired size with the BI interpolation.

   The performance of the proposed method is evaluated qualitatively and the comparison with Bicubic, SR using neighbor embedding, SpNE is given in this section. To evaluate the performance of the proposed method, the reconstructed HR imges of the test LR images and their local magnification results are given in the following figures.

   Fig 2 3X recovery of Plant image using different methods. From left to right the low resolution image; Bicubic; SR using NE; SpNE; the proposed method.

   Fig 3 Local magnification of Plant image. From left to right: Bicubic; SR using NE; SpNE; the proposed method

   TABLE I

   PSNR (IN DECIBELS) OF RECONSTRUCTED IMAGES BY DIFFERENT METHODS.

   Images	BI	SR using NE	SpNE	Proposed
   Girl	31.67	31.72	32.62	33.11
   Leaves	21.85	23.26	23.56	25.26
   Butterfly	22.09	24.13	24.05	26.02
   Parrots	26.48	27.41	27.78	28.91
   Plants	29.72	31.41	31.05	32.03
   Hat	28.27	29.20	29.11	31.07
   Flower	26.22	27.11	27.35	28.96
   Bike	21.82	23.18	22.75	24.85

4. CONCLUSION

Using the idea of neighbor embedding and sigmoid function, a super resolution method is proposed. Method is evaluated quantitatively using PSNR and gives better results compared to other methods. Quality of the reconstructed image is also increased.

REFERENCES

1. S. Park, M. Park, and M. G. Kang, Super-resolution image reconstruction, a technical overview, IEEE Signal Process. Mag., vol. 20, no. 5,pp. 2136, May 2003.

2. C. Staelin, D. Greig, M. Fischer and R. Maurer, Neural network image scaling using spatial errors, HP Laboratories Israel, October 2003.

3. C.B. Atkins, C.A. Bouman and J.P. Allebach, Optimal image scaling using pixel classification, Proceedings of International Conference on Image Processing, 2001, pp. 864867.

4. J. D. van Ouwerkerk, Image super-resolution survey, Image Vis. Comput., vol. 24, no. 10, pp. 10391052, Oct. 2006.

5. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester. Image inpainting, in Proceedings of the Twenty-Seventh Annual Conference on Computer Graphics and Inter-active Techniques (SIGGRAPH 2000), pp. 417424, July 2000.

6. W. T. Freeman, T. R. Jones, and E. C. Pasztor, Example-based super- resolution, IEEE Comput. Graph. Appl., vol. 22, no. 2, pp. 5665, Mar. /Apr. 2002.

7. H. Chang, D.-Y. Yeung, and Y. Xiong, Super-resolution through neighbor embedding, in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jul. 2004, pp. 275282.

8. S. T. Roweis and L. K. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science, vol. 290, no. 5500, pp. 2323 2326, Dec. 2000.

9. J. Yang, J. Wright, T. Huang, and Y. Ma, Image super-resolution via sparse representation, IEEE. Trans. Image Process., vol. 19, no. 11,pp. 28612873, Nov. 2010.

10. X.Gao and K.Zhang,Image Super-Resolution With Sparse Neighbor Embedding, IEEE Transactions on Image Processing, vol. 21, no. 7, July 2012.

11. H. Mohimani, M. Babaie-Zadeh, and C. Jutten, A fast approach for overcomplete sparse decomposition based on smoothed L0 norm, IEEE Trans. Signal Process., vol. 57, no. 1, pp. 289301, Jan.2009.

12. N. Dalal and B. Triggs, Histograms of oriented gradients for human detection, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit, Jun. 2005, pp. 886893.

13. Z. WANG, A. C. BOVIK, H. R. SHEIKH, AND E. P. SIMONCELLI, IMAGE QUALITY ASSESSMENT: FROMERROR VISIBILITY TO STRUCTURAL SIMILARITY, IEEE. TRANS. IMAGE PROCESS., VOL. 13, NO. 4, PP. 600 612, APR. 2004.