Comparison of Image Fusion Technique by Various Transform based Methods

Comparison of Image Fusion Technique by Various Transform based Methods

C. Saranya

ME (Communication Systems) AIHT, Anna University Chennai, India

S. Shoba

Assistant professor (ECE Dept.) AIHT, Anna University Chennai, India

Abstract- The transform based methods are common in pixel level image fusion technique. Image fusion technique is the process to fuse two images and aims to obtain more information in one image by using appropriate fusion ruleswith pixel by pixel basis. Image fusion technique is to combine the information of a variety of images with computer based image processing method.The determination of image fusion technique is to afford information combined from different images, to eliminate redundancy and inconsistency existed between the images. This technique is used to improve the interpretation, accuracy, reliability and utilization of the information with the enhancement of transparency of image information by forming a clear and accurate description of observed target. The various transform based methods consists of discrete wavelet transform (DWT), discrete cosine transform (DCT) and Hybrid transforms (DWT with NSCT). The fused image of decomposed sub bands are obtained by applying inverse transforms. A transform based image fusion technique is employed in order to improve efficiency in an effective manner. By this method better PSNR (peak signal to noise ratio) value with less MSE (mean square error) can be obtained.

Index terms- Image fusion, discrete wavelet transform (DWT), discrete cosine transform (DCT),hybrid transforms, non-sub- sampled contourlet transform (NSCT), Peak signal to noise ratio (PSNR), Mean square error (MSE).

1. INTRODUCTION

   The image fusion technique consists of several application domains. Image fusion technique is used to enhance the quality of information from different modality of images. This technique is used in many application areas and become a common term which is used in medical diagnostics and treatment, microscopic imaging, remote sensing, computer vision and robotic applications. The main applications of image fusion technique is medical diagnosis support system and multi focus imaging system. In medical image fusion technique the multiple images of a patient are merged in order to provide additional information for diagnosis and treatment. The medical image fusion technique is used to merge the information such as positron emission tomography (PET), computed tomography (CT), single photon emission computed tomography (SPECT), and magnetic resonance imaging (MRI). In radiation oncology andradiology the image fusion technique provides a several different determinations. In remote sensing and mapping applications, the multisensor data fusion becomes more common and provides the fusion of panchromatic (PAN) and

   multispectral (MS) images by compromising a highdegree of accuracy and image classification.It affords high spatial and high spectral resolution in a single image is an important issue. The multisensor data fusion provides a tradeoff between spatial and spectral resolution. Thefusion process should preserve all relevant information of original images and it should not introduce any artifacts or contradiction and irregularitiesin fused image. The image fusion technique is used for many reasonslike improving resolution and implementing segmentation.

   The image fusion process can be performed with various levels of image representation are feature level, pixel level, decision level and data level. In pixel level image fusion technique, the fused image can be obtained in which each pixel determines a set of pixels in various source images. The advantage of the fused images are more informative than input image and it also comprises original information. While the pixel level fusion is compared in the decision level fusion and feature level fusion, it is more efficient and easy to implement.

   The image fusion technique is done by taking the average of source images in pixel by pixel basis.Theimage fusion process can be classified in to two domains spatial domain and transform domain. The fusion methods such as Brovey method, Principal component analysis (PCA), Intensity and hue saturation (IHS) and High pass filtering based techniques are fall under the nature of spatial domain. But the spatial domain will produce a spatial distortion in the fused image which is a major drawback. To overcome the drawback of spatial domain the transform based methods with pixel level fusion method is proposed. The transform based methods used are discrete wavelet transform (DWT), discrete cosine transform (DCT) and hybrid transform.The hybrid transform consists of multilevel wavelet decomposition with non-subsampled contourlet transform (NSCT). The transform domain shows a better performance than spatial domain approach.

2. TRANSFORM BASED IMAGE FUSION TECHNIQUES

   1. DISCRETE WAVELET TRANSFORM BASED IMAGE FUSION

      In digital image processing, the discrete wavelet transform is a mathematical tool which is used to detect local

      features of the image. The wavelet transform is used to decompose the two dimensional (2-D) image such as 2-D gray scale image in to different number of frequency components called sub bands. The applications of wavelet transforms are image compression and it transforms the image in to a frequency components. The decomposed sub bands are used for multiresolution analysis with different resolution levels. In DWT the quality of an image can be enhanced by increasing the decomposition levels.

      Figure.1 Schematic representation of DWT based image fusion technique

      DWT is a spatial frequency decomposition and it provides a multiresolution analysis of an image. While applying 2-D DWT to an image, first a 1-D DWT is performed on rows and columns. When it is performed on columns it consists of two sub bands i.e., one low frequency band and one high frequency band. When DWT is performed on rows it will obtain a four sub bands i.e., one set of approximation coefficients and three sets of detailed coefficients. The detailed coefficients in a decomposed image represents the horizontal, vertical and diagonal or approximation components of the image respectively. The approximated and detailed coefficients consists of four sub images which corresponds to the output of low-low (LL), low-high (LH), high-low (HL) and high-high (HH) sub bands.The low frequency decomposed sub band image consists of visible components and whereas the high frequency decomposed sub band image consists of detailed components of image such as texture, lines and edges. The sub band images will achieve image decomposition with desire levels of frequency bands. The schematic representation of figure.1 clearly illustrate the DWT based image fusion technique.

      The detailed steps to perform DWT based image fusion technique are as follows

      Step1:Apply discrete wavelet transform (DWT) to each input images separately.

      Step2:The input images are decomposed in to four sub band images with various levels of decomposition. The decomposed sub band image consists of one low frequency component and three high frequency component of image.

      Step3: The decomposed sub band image consists of transform coefficients can be integrated by applying a certain fusion rule.

      Step4: Finally by performing inverse discrete wavelet transform(IDWT) on the transform coefficients the fused image can be obtained.

   2. DISCRETE COSINE TRANSFORM BASED IMAGE FUSION

      The discrete cosine transform (DCT) is used to separate the image in to 8×8 blocks. The discrete cosine transform is sed to decompose the spatial frequency of image in terms of various cosines. The three frequency components of DCT are low frequency block, middle frequency block and high frequency block. The first block has low frequency block. Second block has middle frequency information and whereas third block consists of high frequency information. The information of approximate coefficients can be viewed in lowFrequency block. The schematic representation of figure.2 clearly illustrate the DCT based image fusion technique.

      Figure.2 Schematic representation of DCT based image fusion technique.

      The image fusion technique based on DCT is to perform a decomposition on each input image and the coefficients are separately fused by using certain fusion rule. Finally the fused image is obtained by performing the inverse discrete cosine transform (IDCT) for the combined high and low frequency coefficients.

   3. HYBRID TRANSFORM

      The hybrid transform consists of multilevel wavelet decomposition (DWT) and non-subsampled contourlet transform (NSCT) with various levels of decomposition. The schematic representation of figure.3 clearly illustrate the hybrid transform based image fusion technique. The hybrid transform first decompose the input image in to the high- frequency coefficient and low frequency coefficients by using one transform (DWT). By selecting the coefficients with largest energy the high frequency coefficients are combined and by using the other transform based image fusion technique i.e., NSCT the low frequency coefficients are combined.

      Figure.3 Schematic Representation of Hybrid Transform Based Image Fusion Technique

      The DWT represents the small features such as angles and textures but it suffers from the drawback of limited directional pattern and lack of correlation of significant wavelet coefficients along discontinuity curves and it produces a poor representation of edges when images have contours and curves. The NSCT consists of multidirectional and multiscale image decomposition scheme with nonsubsampled multiscale pyramids and nonsubsampled directional filter bank. The hybrid transform is used to extract the geometric features of image very well and whereas the contourlet transform represents the long edges. The NSCT is implemented in to two levels and each level contains eight orientations with edge response values.

   4. FUSION METHOD

      The fusion method will govern how the coefficients are incorporated in a suitable way and so that the high quality offused image can be obtained. The fusion method will lead to provide a better image fusion result. The coefficients of low frequency sub image and high frequency sub image are performed with different fusion schemes.

      Averaging method

      The average fusion method is applied for low frequency (LF) bands.It is performed by taking average of pixel values on two input images. The pixel value (x, y)of each image is taken and added. This sum is divided by two to obtain the average. The average value is given to the corresponding pixel of the output image and it is repeated for all pixel values.It is used to extract the clear information and approximate coefficients of image.

      Gradient method

      The gradient fusion method is applied for high frequency bands and consists of detailed coefficients. It is used to extract the texture, lines and edges of image. This method represents the geometrical information of the desired image.

      LDP method

      The local directional pattern (LDP) is a robust feature descriptor, and it computes the edge response values in all eight directions and used to encode the texture of image. The LDP based fusion method is used for high frequency sub bands produced by hybrid transforms. In each

      pixel position it generates a relative strength magnitude and the edge response values are noise insensitive and more illumination of intensity values. The LDP describes the local primitive features such as different types of curves, corners and junctions.

      SF method

      The spatial frequency fusion method is applied for low frequency components i.e., approximate coefficients produced by hybrid transforms. This method is used to extract the coarsest details of two images. The decomposed sub bands of low frequency components are fused by selecting the coefficients which are having maximum spatial frequency among each pixel values.

3. RESULTS

   This section provides the results of image fusion with transform based methods and also evaluates the performance using mat lab with experimental results are presented below.

   1. Input image 1 (b) Input image 2

   1. Fused image by DCT (d) Fused image by DWT

      (e) Fused image by hybrid transform

      Figure.4Multifocus input images (a) focus on clock (b) focus on student (c)-(e) are the fusion results of DCT, DWT and hybrid transform.

      1. CT image of brain (b) MRI image of brain

   The figure.5 (c)-(e) is the result of fused image by DCT,DWT and hybrid transform.DWT provides fused image with better information about soft tissue and bone tissue by combining both the CT and MRI images. The DCT based image fusion technique provides fused image with little distortion. By comparing DWT and DCT with hybrid transform, the hybrid transform has been proved to be an effective one and it provides better information in fused image. The performance analysis based on peak signal to noise ratio (PSNR), Correlation coefficient and mean square error (MSE) is mentioned below.

   Mean Square Error

   Mean square error is used to determine the difference between the input image and the fused image. MSE is the average of square of errors that is mean square multiplied by 255^2 as input image.

   = 1

   ×

   (

   =1 =1

   (c) Fused image of DCT (d) Fused image of DWT

   Where,

   ) 2 (1)

   aij -pixel value at position (i, j) in the input image.

   bij -pixel value at position( i, j) in the fused image.

   Peak Signal To Noise Ratio

   PSNR is a measure of error. The term peak signal to noise ratio is an expression for the ratio between the maximum possible value of a signal and the power of distorting noise that affects the quality of its representation. This is expressed in terms of logarithmic decibel.

   (e) Fused image by hybrid transform

   = 10 log

   255²

   10

   (2)

   Figure.5 Medical input images (a) CT image (b) MRI image (c)-(e) are the fusion results of DCT, DWT and hybrid transform.

   The input images used in figure.4 are multifocus images such as in figure.4(a) the student is in out of focus and

   Correlation Coefficient

   It defines the similarity structures between the input and fused image. The higher value of correlation means that more information is preserved. The correlation coefficient is defned by

   ( × )

   in figure .4(b) the clock is in out of focus. Both the student

   and clock are in focus images in figure.4(c)-(e).

   =

   =1 =1

   ()²

   (3)

   The input images used in figure.5 are medical input

   Where,

   =1

   =1

   images such as computed tomography (CT) image and magnetic resonance imaging (MRI). Both the CT and MRI images are tomography scanning images and they have different features. The computed tomography provides a visual description of bone tissue, whereas magnetic resonance imaging provides the better visualization of soft tissues and it can be shown in figure.3 and figure.4

   a-difference between input image and its overall mean value. b-difference between fused image and its overall

   mean value.

   Table.1Performance measures of transform based methods

   TRANSFORMS USED	FUSION METHOD	MSE	PSNR	CORRELATION COEFFICIENT
   DWT	Averaging and Gradient method	5.6399	40.6181	0.9325
   DCT	Averaging and Gradient method	7.3126	38.3281	0.8867
   HYBRID TRANSFORM	Local directional pattern and spatial frequency method	3.7412	42.4007	0.9878

   The values of PSNR,MSE and correlation coefficient is shown in table 1.For a better quality of fused image, the peak signal to noise ratio should have a larger value. The hybrid transform provides a better fused image with high PSNR value and less MSE.

4. CONCLUSION AND FUTURE WORK

In transform based image fusion technique hybrid transform (NSCT with DWT) has been proved to be more effective one. In wavelet based methods the fused quality of the image is determined by the fusion rule. The traditional fusion methods have two serious drawbacks, one is selecting the coefficients with the same rule and another is selecting the coefficients independently. To overcome this hybrid transform with local directional pattern and spatial frequency fusion method is presented. For fusing the coefficients in discrete wavelet transform and discrete cosine transform based image fusion process the low and high frequency bands are taken separately. The performance of the LDP and SF based fusion method is better than the other methods. In future this can be resolved by selecting the low and high frequency coefficients with same fusiion method and use decoding process to obtain the better quality of fused image.

REFERENCES

1. Andreas Ellmauther,Carla L.Paglari and Eduardo A.B.da Silva (March 2013), Multiscale image fusion using the undecimated wavelet transform with spectral factorization and nonothogonal filter banks

   ,IEEE trans on Image processing, Vol.22,No.3,pp no 1005-1017.

2. Shutao Li and Bin Yang (September 2010), Hybrid multiresolution method for multisensor multimodal image fusion ,IEEE Sensors Journal, Vol.10,No.9,pp no 1519-1526.

3. Jorge Nunez,Xavier Oazu,Octavi Fors,Albert prades,Vicenc pala and Roman Arbiol (May1999), Multiresolution based image fusion with Additive wavelet decomposition ,IEEE trans on Geoscience and remote sensing, Vol.37, pp no. 1204-1211.

4. Mrityunjay Kumar and Sarat Dass (September 2009), A total variation based algorithm for pixel level image fusion ,IEEE trans on Image processsing, Vol.18,No.9,pp no 2137-2143.

5. Jean-Luc Starck,Jalal Fadili and Fionn Murtagh (February 2007), The undecimated wavelet decomposition and its reconstruction ,IEEE trans on Image processing, Vol.16,N0.2,pp no 297-309.

6. Petrovic V.S and Xydeas C.S (February 2004), Gradient based multiresolution image fusion, IEEE trans on image processing, Vol.13,N0.2,pp no 228-237.

7. Li.s and yang.B (September 2010), Hybrid multiresolution method for multisensor multimodal image fusion ,IEEE Sensors Journal, Vol.10,No.9,pp no 1519-1526.

8. Pajares.G and Cruz.J (March 2004), A wavelet based image fusion tutorial ,Pattern recog, Vol .37,No.9,pp no 1855-1872.

9. Gopalakrishnan Nair T.R and Richa Sharma (2013), Accurate Merging of Images for Predictive Analysis using Combined ImageIEEE trans on image processing, Vol. 10 No .14, pp no 1865-1892.

10. LIU Shan-shan, Zhang Xiao-he and Zheng Ai (2008), Image Fusion Algorithm Based onWavelet Sparse Represented Compressed SensingIEEE trans on image fusion,Vol. 10, No.9, pp no 1519-1526.

11. Ding Li,H and Chong Zhao (2007), Multi-focus Image Fusion Using Wavelet Based on Contourlet Transform IEEE trans On image fusion, Vol 5,No.7,pp no 203-220.

12. Lallier E. and Farooq M. (July 2000) A real time pixel-level based image fusion via adaptive weight averaging IEEE trans on Image processing, Vol. 2, pp no WeC3/3WeC3/13.

13. Liu Shan-shan, Zhang Xiao-he and Zheng Ai,(April 2008), Image Fusion Algorithm Based on Wavelet Sparse Represented Compressed SensingIEEE trans on Image fusion, Vol. 10, No.9, pp no 1519-1526.

14. Ding Li. H and ChongZhao (2007), Multi-focus Image Fusion Using Wavelet Based on Contourlet Transform IEEE trans on Image fusion, Vol. 5, No.7, pp no 203-220.

15. Ik-Hyun Lee (September 2013), Accurate registration using adaptive block processing for multispectral images, IEEE trans on circuits and systems for video technology, Vol.23 No. 9 pp no 1491-1501.

16. Liu Cao,Longxu Jin,Hongjiang Tao Guoning Li and Yanfu zhang (2015), Multi-focus image fusion based on spatial frequency in discrete Cosine transform domain,IEEE trans on Signal processing letters, Vol.22 No.2 pp no 220-224.

17. Zhou.J,Civco D.L and Silander J.A (1998), A wavelet transform method to merge landsat TM and SPOT panchromatic data

    ,Int.J.Remote sensing, Vol.19,No.4,pp no 743-757.

18. Wald.L,Ranchin.T and Mangolini.M (1997), Fusion of satellite images of different spatial resolutions:Assessing the quality of resulting images ,IEEE trans on Remote sensing, Vol.63,No.6.pp no 691-699.

19. Mallat.S (July 1989), A theory for multiresolution signal decomposition:The wavelet representation ,IEEE trans.Pattern Anal.Machine Intell, Vol.11,pp 674-693.