LWT based Image Super Resolution

LWT based Image Super Resolution

Gini Christopher

Department of Electronics and Communication Marian Engineering College

Trivandrum, India

Minnu Jayan.C

Department of Electronics and Communication Marian Engineering College

Trivandrum, India

AbstractThis paper presents an effective approach to recover a high resolution image from a single low resolution input image. In this technique lifting wavelet transform and stationary wavelet transform is used to increase the spatial resolution .The wavelet domain filters support to model the regularity of natural images while the edge details of image get sharper while up sampling. An iterative back projection method is used to reconstruct the high resolution image in an efficient iterative manner. Experimental results demonstrate that the proposed approach is very effective in increasing resolution compared to state-of–art super-resolution algorithms.

Keywords S uperresolution; lifting wavelet transform; stationary wavelet transform; backprojection; reconstruction

1. INTRODUCTION

   In most electronic imaging applications like medical imaging ,image/video coding and resizing , remote surveillance, high definition television broadcasting, and face recognition high-resolution images are required . Due to certain factors like storage limitation ,limited computational power , cost of camera and insufficient bandwidth, high resolution images cannot be captured directly. In such situations super resolution algorithms play a major role . The basic idea of super resolution is to get high resolution image from a low resolution image which looks as it has been captured from a sensor having the desired resolution. Super resolution algorithm can be classified into two,

   • Single image super resolution

   • Multiframe super resolution

   Out of the above single image super resolution is the most simplest and can be implemented in either spatial domain or wavelet domain. Super resolution algorithm was first proposed to work on multiple images of same scene and generate single image [1]. Now a days, algorithms for real time and single image super resolution have been developed. The spatial domain based algorithms are mostly discussed and studied due to the relation between high resolution and low resolution images. Recent works on single image super resolution has been done using patch based up sampling ,texture hallucination and example based super resolution[2,3]. Algorithms that use edge priors to reconstruct sharp images result in over smoothness and blurrness in certain regions [4]. Freeman et al. [5] used patch-based image model in which training dataset is used which generates good result but it is

   possible for those images for which training data is available and generates some noise also. Simple interpolation techniques like bilinear interpolation and pixel replication up scale an image without considering any details of input image. These techniques perform well in smooth region but high frequency detais such as edges and complicated textures get blurred.The challenge in wavelet domain based image interpolation is to estimate unknown coefficients of three high frequency sub bands. Wavelet Zero-Padding is the basic interpolation method used in wavelet domain. Here a low resolution image is multiplied by a scaling factor S, which acts as the top left quadrant (LL) of final high resolution image.Zeros are padded in rest of the three high-resolution image (HH, HL and HH) quadrants. Temizel [6] combined the directional cycle spinning method with Wavelet Zero-Padding (WZP) interpolation method where shifted image information was exploited in the reconstruction stage. Zhang [7] proposed a soft decision method to compute the parameters of Autoregressive Model (AM) adaptively. This approach can obtain high quality of the upscaled images, but its corresponding computational complexity inhibits its possibility to implement on hardware.Dong et al. [8] proposed a novel non-local IBP algorithm for image enlargement which incorporates adaptively the non-local information into the IBP process so that the reconstruction errors can be minimized. In[9], Liang et al. proposed an improved NLIBP fast algorithm in which edges are detected on the initially interpolated image.Time consumed is greatly reduced by the use of non- local filter in the modifying process. J. Wang et al.[10] proposed the new an Estimated High Frequency Compensated (EHFC) algorithm for super resolution images. It effectively combines the IBP method with compensated high frequency models according to different applications. Dai [11] proposed the alpha-matting model to obtain a MAP decomposition of any local image patch into background and foreground components and reconstruct the discontinuity between them. High-resolution images with good visual qualities are generated, but computational cost very is high.

   In the wavelet-domain based techniques for image interpolation, the low wavelet sub band of the wavelet- transformed high resolution image is treated as low resolution image. The most difficulty is to estimate the unknown coefficients of the other three high wavelet sub bands.Woo

   1. proposed the statistical models, particularly Gaussian Mixture (GM) model in which the inter-scale correlations between the high wavelet coefficients and the low wavelet coefficients are exploited. . Dai et al. in [13], uses bilateral filter with IBP method, which uses single LR image.

      In this paper, LWT based single image super resolution is proposed to upscale an image . The advantages of both spatial- domain based and wavelet-domain based algorithms are exploited. At the same time an iterative back-projection method is used to reconstruct the high resolution image in an efficient iterative manner.

2. PROPOSED METHOD

   The low resolution image IL can be obtained by multiplying with a scaling factor S,which work as top left quadrant (LL) of final high resolution image. Thus obtained input low resolution image is decomposed into different frequency sub bands by lifting wavelet transform using Haar as lift wave. Four frequency sub bands are generated by LWT, out of which three are high-frequency and one is low-frequency sub band. The high frequeny sub bands contain detail coefficients which are essential in the reconstruction of the final high resolution image.

   The stationary wavelet transform which generates various frequency sub bands ,high-frequency sub bands contain horizontal, vertical and diagonal detail coefficients of input image.The sub bands generated by SWT has the size same as that of the input image.Initial interpolation of high-frequency sub bands generated using LWT is required because LWT uses sampling that generates frequency sub bands half of the size of input image. So high-frequency sub bands generated by LWT are initially interpolated with factor of 2 . These interpolated high frequency sub bands are adjusted by adding high-frequency sub bands generated using SWT. We combine the high wavelet sub bands (LH,HL and HH) achieved by both LWT and SWT with the low wavelet band (LL) used in LWT. This combination result in initialization of detailed high frequency sub bands which is essential for iterative back projection.

   Based on this pre-process, we are able to refine the wavelet coefficients in different sub bands in the later iterations. The sub bands are combined using inverse lifting wavelet transform. After up- sampling the image IH(t) can be blurred little bit, by using a Gaussian filter which merely work as smoothing kernel. As the blurred effect is very low o ignorable Gaussian filter is applied only once. In this stage of algorithm error E(t) is calculated which is defined as the difference between original low-resolution image IL and down sampled image after Gaussian filtering (IH(t)*G) k.

   E(t) = IL (IH(t) * G) k (1)

   Where k is the down sampling factor and G is the point spread function.

   This is the most important part of algorithm because the error that we find in this stage is used as the correction parameter in getting super resolved image and also used for refining coefficients of sub bands. Finally high resolution image is obtained by back projecting the error as:

   IH(t + 1) = E(t) k *p + IH(t). (2)

   Where p is the back projection kernel and k is the downsamplig factor.

3. MAIN ALGORITHM

   Fig. 1 is the block level representation of the proposed algorithm and the following introduces the details in each step. The first step is down sampling to get low-resolution image. The low-resolution image can be created by taking the high-resolution and down sampling by a factor of 4.The high resolution image is first divided into 2×2 blocks and the new pixel value of down sampled image IL is obtained by taking the mean value of each block.

   Next we have to up sample the image which is the second step of the algorithm. The initial high-resolution image IH(0) is achieved by the synthesis of wavelet coefficients from high- resolution image interpolated by using LWT and SWT. The mother wavelet used is HAAR because it is computationally fast. There are several other wavelets such as sym; db4 etc. but the computation time required is more. The proposed up sampling scheme is shown in fig 2.

   The analysis filter bank which consist of low pass and high pass filters at each decomposition stage can be used for signal decomposition and split signal into two bands. The coarse information is fetched by the low pass filter (equivalent to an averaging operation) whereas the high pass filter fetch detail information (equivalent to a differencing operation) of the image.

   For 2-D transform, filtering of the image is done along the x-dimension using low pass and high pass analysis filters and decimated by a factor two, followed by filtering of the sub- image along the y-dimension and decimated by two. After one level of decomposition the resultant image has been split into four bands LL, HL, LH, and HH,. The LL band is again subject to decomposition. This process is called pyramidal decomposition of the image. Reversing the above procedure results in the reconstruction of the image.

   Finally all four-sub bands are interpolated by a factor K/2 and inverse LWT is taken to get the up sampled image with size km x kn .

   G

   d

   The third step is Gaussian filtering. After the up sampling process the image may look little bit blurred. So Gaussian filter can be used to reduce rise and fall time of the step function of the input. Since Gaussian filter is a smoothing kernel, it is applied once in the first stage of iteration .The filtered image Ih , is then down-sampled. It is same as that of step 1 ie, down-sampled by averaging every 4 pixels, forming Il (t),where t means the tth iteration.

   G

   Reconstruction is the most important part of the algorithm.In this stage of algorithm error is calculated between original low-resolution image IL and down sampled image Ih and is termed as reconstruction error E(t).The error that we find in this stage is used as the correction parameter for refining coefficients of sub bands. After three iterations error becomes so small that it can be neglected. Observed error is given by (1).

   Low Resolution Input IL

   LWT-SWT based up sampling

   Gaussian filtering

   Down sampling

   Fig1. Block level representation of the algorithm.

   Next stage is up sampling the error.Here the error is up sampled using the same technique as described in step 2.The purpose of upsampling the error matrix E(t) is to meet size of high resolution image.The resulting error matrix is denoted as EH(t). Finally error matrix EH(t) generated is added with high- resolution image generated in step 3 as follows.

   IH(t + 1) = E(t) 4 + IH(t). (3)

   IH(t+1) is the high resolution image in the next iteration. The up sampling operator is given by 4 . This process is called back projection. The above procedure is repeated until we get super resolved image.

   Low resolution input

   Bicubic inter- polation

   SWT

   LWT

   Combining the high frequency bands

   ILWT

   Initial high resolution image IH(0)

   Fig 2. Proposed Up-sampling Scheme

4. EXPERIMENTAL RESULTS

   Computing reconstruction error

   Up sampling of error using LWT-SWT

   scheme

   Back projecting the error

   High resolution image

   The proposed algorithm is tested with test images taken from USC-SIPI image database[16] and experiments shows that it works well for all type of images without any complexity.

   Fig.3(a) Fig.3(b)

   Fig.3(c) Fig.3(d)

   Fig.3:(a)lena input image of resolution 128 ×128 (b)proposed method output 512×512 (c)output using [17] 512×512 (d)bicubic interpolation 512×512.

   Fig.3(a) is an input image of resolution 128×128, Fig.3(b) represents the 512× 512 resolution output obtained by the proposed algorithm, Fig.3(c) represents the 512× 512 output obtained using [17] and Fig.3(d) shows the 512× 512 output using bicubic interpolation. By analyzing the results obtained using various methods,we can see that edge information is more clear in the proposed one. Fig.4 shows the edge information obtained from the image of lena which is then backprojected in the reconstruction stage of the algorithm. Fig.5 shows the results obtained using various techniques

   when pepper image is given as input. By analysis of PSNR, SSIM,VIF and entropy values ,the proposed method is more superior in providing better image quality.

   Fig.4 :Reconstruction error

   Fig.5(a) Fig.5(b)

   Fig.5(c) Fig.5(d)

   Fig.5:(a) input image of resolution 128 ×128 (b)proposed method output 512×512 (c)output using [17] (d)bicubic interpolation

   The algorithm can be extended to enhancing the resolution of satellite images ,which gives results with better PSNR values. Fig.6 shows the resolution enhancement of satellite image by the proposed method and other conventional state- of – art methods. This method can also be applied to medical images ,where resolution is a major requirement.Fig.7 shows the PSNR plot of a set of images using different methods.

   Fig 6(a) Fig 6(b)

   Fig 6(c) Fig 6(d)

   Fig.6:(a)input satellite image of resolution 128 ×128 (b)proposed method output 512×512 (c)output using [17] 512×512 (d)bicubic interpolation

   512×512.

5. PERFORMANCE EVALUATION

   The image quality can be measured using several techniques and metrics that gives the perceived image degradation objectively and automatically .There are two categories of quality assessment: Full Reference Method (FR) and No Reference Method(NR).In FR quality assessment method, the quality of the image is evaluated by comparing with a reference image which is assumed to be perfect. Some of FR methods are PSNR, SSIM,VIF etc. In NR method there is no reference image. Some of the NR methods are Entropy,Standard deviation etc.

   1. Peak Signal to Noise Ratio

      The PSNR is defined as the ratio of maximum possible power of the signal to the power of corrupting noise.

      PSNR = 20 · log 10(MAX i / MSE)

   2. Entropy

      Entropy is defined as the information contained in an image. High value of entropy indicates that the reconstructed image is perfect.

      E = P*log2P

   3. Structural Similarity Index

      Structural Similarity Index is defined as a measure of similarity between two images. Its value ranges from 0 to 1.It is an effective and consistent parameter .

   4. Visual Information Fidelity

   VIF is defined as the ratio of distored image information to reference image information.

   QUALITY MEASURES OF RECONSTRUCTED IMAGE
   PARAMETER	LENA	PEPPERS	SATELLITE IMAGE
   PSNR	34.8163	33.1767	33.7652
   ENTROPY	7.7101	7.7502	7.2493
   SSIM	0.9579	0.9660	0.9777
   VIF	0.8881	0.8690	0.8819

   TABLE 1: Performance measure for set of images

   Fig 7. Plot of PSNR for various techniques

6. CONCLUSION

Resolution enhancement has become a mandatory in many applications. Proposed method uses lifting wavelet transform along with stationary wavelet transform to generate high frequency sub bands which are refined using an iterative back projection method. The lifting scheme builds a new wavelet, with improved properties, by adding a new basis function and does not require auxiliary memory. The LWT has advantage over DWT such as the transform can be modified locally while preserving invertibility. From the results it is possible to say that the proposed method gives much better values of parameters such as PSNR, entropy, SSIM, VIF and the computational time required is low. This method can also be extended to super resolution of videos.

REFERENCE

1. S. C. Park, M. K. Park, and M. G. Kang, Super-resolution image reconstruction: A technical review, IEEE Signal Processing Mag., vol. 20, pp. 2136, May 2003.

2. D. Glasner, S. Bagon, and M. Irani. Super-resolution from a single image. In Proc. ICCV, 2009.

3. Yoav HaCohen,Raanan Fattal ,Dani Lischinski,"Image Up sampling via Texture Hallucination",IEEE International Conference on Computational Photography (ICCP 2010).

4. R. Fattal. Image upsampling via imposed edge statistics. ACM Trans.

   Graph., 26(3):95, 2007

5. W. T. Freeman, T. R. Jones, and E. C. Pasztor. Example- based super- resolution. IEEE Comput.Graph. Appl., 22(2):5665, March 2002.

6. Temizel and T. Vlachos, Image resolution up scaling in the wavelet domain using directional cycle spinning, J. Electronic Letters, vol. 41, pp. 119121, 2005.

7. X. Zhang and X. Wu, Image interpolation by adaptive 2-d autoregressive modeling and soft-decision estimation, Image Processing, IEEE Transactions on, vol. 17, no. 6, pp. 887896, june 2008.

8. W. Dong, L. Zhang, G. Shi, and X. Wu, Nonlocal backprojection for adaptive image enlargement, in Image Processing (ICIP), 2009 16th IEEE International Conference on,nov. 2009, pp. 349 352.

9. X. Liang and Z. Gan, Improved Non-Local Iterative Back Projection Method for Image Super-Resolution, Sixth International Conference on Image and Graphics (ICIG), 2011 .

10. J. Wang, K. Liang, S, Chang, and P. Chang, Super-Resolution Image with Estimated High Frequency Compensated Algorithm, IEEE Conf. Communication and Information Technology, 2009, pp. 1039-102.

11. S. Dai, M. Han, W. Xu, Y. Wu, and Y. Gong, Soft edge smoothness prior for alpha channel super resolution, in Computer Vision and Pattern Recognition, 2007. CVPR 07. IEEE Conference on, june 2007, pp. 1 8.

12. D. H. Woo, I. K. Eom, and Y. S. Kim, Image interpolation based on inter-scale dependency in wavelet domain, in Image Processing, 2004. ICIP 04. 2004 International Conference on, vol. 3, oct. 2004, pp. 1687

    1690 Vol.3.

13. S. Dai, M. Han, Y. Wu, and Y. Gong, Bilateral backprojection for single image super resolution, in Multimedia and Expo, 2007 IEEE International Conference on, july 2007, pp. 1039 1042.

1. X. Li and M. T. Orchard, New edge-directed interpolation,IEEE Transactions on Image Processing, vol. 10, pp. 15211527, 2001.

2. A. Weber, USC-SIPI Image Database. [Online]

   Available:http://sipi.usc.edu/database/database.php

3. Shen-Chuan Tai, Tse-Ming Kuo,Chon-Hong Iao,Tzu-Wen Lia,"A Fast Algorithm for Single-Image Super Resolution in both Wavelet and Spatial Domain",2012 International Symposium on Computer,Consumer and Control,IEEE.