A Novel Image Fusion Technique using Dual Tree Complex Wavelet Transform based Laplacian Pyramidal Decomposition

A Novel Image Fusion Technique using Dual Tree Complex Wavelet Transform based Laplacian Pyramidal Decomposition

K. Chandana

M. Tech student Dept. of EEE

SVU College of Engineering Tirupati, India

1. Usha Reddy Asst. Professor Dept. of EEE

   SVU College of Engineering Tirupati, India

   Dr. T. Ramashri

   Professor Dept. of ECE

   SVU College of Engineering Tirupati, India

   Abstract Image fusion is the process of combining the information from the multiple images of the same scene, and the fused image retains the most desirable information and characteristics of each input image. The problem of spectral distortion with spatial fusion methods is overcome by employing multi-scale transformation methods, like Wavelet Transform based image fusion techniques. However, the classic Discrete Wavelet Transform also has its own limitation such as, shift variant, poor directionality, which can be overcome by employing complex wavelet techniques. In this paper, image fusion technique using Dual Tree Complex Wavelet Transform based Laplacian Pyramid Decomposition, has been proposed. Initially the images to be fused are subjected to the Dual Tree Complex Wavelet Transform decomposition, and thus obtained approximation wavelet coefficients are processed with the Laplacian Pyramidal Decomposition to get the fused image. The proposed dual tree complex wavelet transform based image fusion technique is compared with the existing wavelet based fusion technique, and it is found that proposed technique outperform over the existing techniques.

   Keywordat: Image Fusion, Laplacian Pyramidal Decomposition (LPD), Discrete Wavelet Transform (DWT), Dual Tree Complex Wavelet Transform (DTCWT)

   1. INTRODUCTION

      Image fusion is a tool that serves to combine multisource imagery by using advanced image processing techniques. Specifically, it aims at the integration of disparate and complementary data in order to enhance the information apparent in the images, as well as to increase the reliability of the interpretation. This leads to more accurate data and increased utility. In addition, it has been stated that fused data provides for robust operational performance such as increased confidence, reduced ambiguity, improved reliability and improved classification [3, 7, 9].

      Image Fusion had its beginning with the concept of simply averaging the intensities of the corresponding pixels of the set of input images, thus producing a fused image. A lot of advancements have happened in the field of image fusion since then employing advanced methods like Discrete Wavelet Transforms and Pyramidal Methods to fuse images. Image fusion methods can be broadly classified into two – spatial domain fusion and transform domain fusion [7, 9]. The fusion methods such as averaging method, Brovey method, principal component analysis (PCA) and high pass filtering based technique are examples of spatial domain fusion methods. Here

      the high frequency details are injected into upsampled version of MS images [2].

      The disadvantage of spatial domain approaches [8, 9] is that they produce spatial distortion in the fused image. Spectral distortion becomes a negative factor while we go for further processing, such as classification problem, of the fused image. The spatial distortion can be very well handled by transform domain approaches on image fusion. The multiresolution analysis has become a very useful tool for analyzing remote sensing images. The discrete wavelet transform has become a very useful tool for fusion. Some other fusion methods are also there, such as pyramid based, curvelet transform based etc. These methods show a better performance in spatial and spectral quality of the fused image compared to other spatial methods of fusion.

   2. A BRIEF REVIEW OF LAPLACIAN PYRAMIDAL DECOMPOSITION AND DISCRETE WAVELET TRANSFORM

      1. Laplacian Pyramidal Decomposition

         Laplacian Pyramid [5, 6] is used to decompose the image into smaller images corresponding to different frequency bands. Each band of the Laplacian pyramid is the difference between two adjacent low-pass images [I0, I1, . ., IN] the Laplacian pyramidal decomposition is obtained using

         I0 = I Ik+1 = R(Ik)

         = (+1) . . . . . (1)

         where, R(Ik) is the reduced size image obtained after smoothing Ik with the low pass filter H(w), and (+1) is the expanded image obtained after smoothing Ik+1 with the low pass filter G(w). In the similar manner, the original image can be reconstructed back by using

         = + (+1). . . . . (2)

         Figure 1. shows the filters and sampling steps used to computethe pyramid, and to then reconstruct the image from the transformcoefficients. The figure illustrates the Laplacian Pyramidal decomposition of the image into its different frequency bands of information, and the reconstruction of original image from its various frequency bands of information.

         limitations of the standard DWT are overcome in DTCWT [1] because of its analyticity.

         The Hilbert transform signal aids in the construction of complex signal, for given real valued signal, and it is defined as:

         () = () + () . . . . . (4)

         where, g(t) is the Hilbert transform of f(t) and denoted as {f(t)} and j = (-1)1/2.Signal g(t) is orthogonal to f(t). In the time domain, g(t) can be represented as

         () = {()} = 1 ()

         . . . . . (5)

         Figure 1. Laplacian Pyramidal Decomposition and Reconstruction for two

         levels

      2. Discrete Wavelet Transform

      The simplest wavelet transform for multi-dimensional digital data is the critically-sampled separable wavelet transform. As the image is a two dimensional array of data, it is subjected 2D Discrete Wavelet Transform [2, 8, 9] to compute its approximation (scaling function) coefficients and detail (wavelet function) coefficients, at different level of decomposition. The figures shown below illustrate the 2D wavelet decomposition of an image.Each decomposition breaks the parent image into four child images. Each of such sub-images is of one fourth of the size of a parent image. The sub-images are placed according to the position of each sub- band in the two-dimensional plane. The structure of synthesis filter-bank follows the reverse implementation of analysis filter bank but with the synthesis filters.Here, h0 and p are the FIR low pass and high pass filters of the wavelet used for decomposition. The LL band coefficients are called approximation coefficients (a), and the remaining bands LH, HL and HH band coefficients are called as the detail coefficients (d1, d2 and d3).

      Figure 2. Single level analysis filter bank for 2-D DWT

      Although the standard DWT is a powerful tool, it is suffering from the limitations [1] of shift invariant and poor directionality.

   3. DUAL TREE COMPLEX WAVELET TRANSFORM (DTCWT)

      Complex wavelet Transform is a powerful tool in signal and image analysis, where most of the properties of the transform follow from the analyticity of the wavelet transform. Complex wavelet transforms use complex valued filtering (analytic filter) that decompose the real and complex signal into real and imaginary parts in a transform. The

      The Dual Tree Complex Wavelet Transform (DTCWT) comprises of two trees for the decomposition and reconstruction; one tree consists of real part of FIR low pass and high pass filters, and the other consists of imaginary part FIR low pass and high pass filters. The key difference between DWT and DTCWT is that all the real filters are replaced with analytic filters to have complex solution [].

      Figure 3(a) Analysis Filter bank for 1D DTCWT

      Figure 3(b) Synthesis Filter bank for 1D DTCWT

      Figure 3 shows 1-D analysis and synthesis filter banks spanned over three levels. It is evident from the filter bank structure of DTCWT that it resembles the filter bank structure of standard DWT with twice the complexity. It can be seen as two standard DWT trees operating in parallel. One tree is called as a real tree and other is called as an imaginary tree. The form of conjugate filters used in 1-D DT-DWT is given as: (hx + j gx). Here, hx is the set of filters {h0, p}, and gx is the set of filters {g0, g1} both sets in only x-direction (1-D). The filters h0 and p are the real-valued low pass and high pass filters respectively for real tree. The same is true for g0 and g1 for imaginary tree.

      For the filter bank structure, shown in figure (3), let h0 and p represent CQF pair [1]. That is,

      p(n)=(-1) (1-n) h0(1-n) . . . . . (6)

      and, in Z-transform domain

      H0(Z) H0(1/Z)+ H0(-Z) H0(-1/Z) = 2

      H1(Z)=1/Z H0(-1/Z) . . . . (7)

      The scaling function h(t) and wavelet g(t)are defined similarly for the imaginary tree. g(t) is the Hilbert transform of h(t),

      The Spatial Frequency parameter can be used to reflect the clarity of an image. Without regard to noise, the large value of SF demonstrates that the image is sharp.

      1. Discrete Wavelet Transform Based Image Fusion Technique

         g(t)= H{ h(t) } . . . . . (8)

         The important properties of DTCWT [1] for the comparison with standard DWT are Shift-sensitivity, Directionality.

         1. Shift Invariance: DTCWT has approximate shift- invariance, or in other words, improved time-shift sensitivity in comparison with standard DWT. The reconstructed details at various levels and approximation at the last level have almost uniform shifts for the time- shifted unit step functions.

         2. Directionality:Standard DWT offers the feature selectivity in only 3 directions with poor selectivity for diagonal features .whereas DTCWT has 12 directional wavelets (6 for each of real and imaginary trees) oriented at angles ± 15 , ± 45, ± 75.

   4. IMPLEMENTATION OF IMAGE FUSION ALGORITHMS Even though several algorithms have been proposed for

      DWT is one of the efficient tools for the signal and image processing techniques. As the DWT adapts both the spatial and frequency domain properties, it is more suitable than the spatial and frequency domain based techniques for image fusion. The algorithm initially decomposes into a series of wavelet subbands using Discrete Wavelet Transform (DWT).

      1. The approximation wavelet coefficients of both the images are further decomposed using Laplacian Pyramidal Decomposition and thus coefficients are fused to a single subband using maximum selection rule.

      2. The spatial frequency parameter is calculated for all the detail coefficients of both images, and the corresponding detail coefficients of the two images are fused into a single subbands using SF parameter [10].

         (, ) = { (, )((, )) ((, ))

         combining various features of images, the image fusion field, still has not reached its maturity. This work focuses on both these requirements and proposes a method that integrates the

         (, ) ((, )) < ((, ))

         . . . . . (10)

         Laplacian pyramid algorithm, wavelets, complex wavelets, and spatial frequency [10].

         Figure 4. Block Diagram of Proposed Image Fusion Technique.

         Figure 4. represents the schematic procedure for the implementation of image fusion technique using DWT/DTCWT based Laplacian Pyramidal decomposition and Spatial Frequency (SF) parameter.The parameter Spatial Frequency (SF) [10] measures the overall activity level in an image. For M × N image F, the spatial frequency is calculated as follows.

         (, ) = (, )2 + (, )2 . . . . . (9)

         where, RF(x, y) and CF(x, y) are the row and column frequency of the image pixel (x, y), and they are given as

         (, ) = (, ) (, 1)

         (, ) = (, ) ( 1, )

         where, da(x, y), db(x, y) and df(x, y) are the detail

         coeffcients of the images A and B, and fused detail coefficients at the particular decomposition level.

      3. The fused image is obtained by computing the Inverse Discrete Wavelet Transform (IDWT) of the fused approximation and detail coefficients.

      As the Laplacian Pyramidal Decomposition extracts the features and structures of the image, the mixing of this feature with DWT enhances the image fusion performance both quantitatively and qualitatively. Spatial frequency is the parameter which fuses the details of DWTs of the both images effectively when compared to direct fusion rules, such maximum, minimum, average etc.

      1. Dual Tree Complex Wavelet Transform (DTCWT) based Image Fusion Technique

      This technique is similar to the DWT based image fusion technique, but the difference is wavelet decomposition. In the case of DWT based image fusion technique, the images are subjected to Discrete Wavelet decomposition, whereas in the case of DTCWT based image fusion techniques, the images are decomposed using Dual Tree Complex Wavelet Transform. The steps in this image fusion techniques are:

      1. Both the images are subjected to Dual Tree Complex Wavelet Transform to get the approximation and detail coefficients.

      2. The approximation and detail coefficients are fusing Laplacian Pyramidal decomposition and SF parameter respectively, as in the case of DWT based image fusion technique.

      3. The fused image is obtained by computing Inverse Dual Tree Complex Wavelet Transform (IDTCWT) of the fused approximation and detail coefficients.

         This techniques is almost similar to the DWT based image fusion technique. The limitations of DWT such as, shift variant and poor directionality are overcome in Dual Tree Complex Wavelet Transform, hence DTCWT outperforms better than DWT for image processing applications like image fusion, image denoising, etc. The important thing to be mentioned here is that DTCWT has the enhanced directional features, hence its performance is better than that DWT based technique.

         The performance of the image fusion is measured by evaluating image quality parameters [4] such as Mean Square Error (MSE), Peak Signal to Noise Ratio (PSNR), Normalized Cross Correlation etc.

         Mean Square Error (MSE): Mean square error is one of the most commonly used error projection method where, the errorvalue is the value difference between the actual data and the resultant data. The mean of thesquare of this error provides the error or the actual difference between the expected/ideal results tothe obtained or calculated result. The MSE of the two image A and B of M × N dimensions, is given by

         Figure 5(a) Image 1

         = 1

         =1

         =1

         ( )2. . . . . (11)

         Peak Signal to Noise Ratio (PSNR): It is defined as log of the ratio between the square of thepeak value to the Mean Square Error multiplied to the value 10. This basically projects the ratio ofthe highest possible value of the data to the error obtained in the data. For 8 bit images, the peak value R = 28

         1 = 255.

         = 1010

         ( 2 ). . . . . (12)

         Normalized Correlation Coefficient (NCC): This is the ratio value between net sum of the correlated of the expected and obtained data and the expected data. It will be the ratio between the net sum of the multiplied values of the expected and the obtained values and the net sum of the squared expected values.

         (×)

         = =1 =1 . . . . . (13)

         =1

         =1

         2

   5. RESULTS AND DISCUSSION

      In this work, the DWT and DTCWT based image fusion techniques are tested with a set of images: Color Text. The performance ofthe image fusion techniques is evaluated with the help of the image quality metrics MSE, PSNR and NCC.

      Initially, the images to fused are selected, and each image is decomposed using DWT (sym5, db5) and DTCWT separately. Thus obtained approximation coefficients are fused Laplacian Pyramidal decomposition and detail coefficients are fused based on the spatial frequency parameter. The fused approximation and detail coefficients are subjected to inverse DWT or inverse DTCWT in order to get the fused image. The fused image quality is judged by the image performance metrics PSNR, MSE and NCC with respect to a reference image.

      Figure 5(b) Image 2

      Figures 5(a) and 5(b) shows the input images to be fused. In the Image 1 the letters DO are not visible clearly, and the letters YOUR is not visible clearly in the Image 2. The blurred text in both images is fused with the other to get good clarity and visibility.

      Figure 5(c) shows the fused image obtained using DWT (sym5 wavelet). In this image, both the words DO and YOUR are visible clearly with good quality.

      Figure 5(d) shows the fused image obtained by using DTCWT based image fusion technique. On comparing the Figures 5(c) and 5(d), it is evident that 5(d) is clearer than 5(c), because Dual Tree Complex Wavelet Transform is advantageous over Discrete Wavelet Transform, by overcoming the DWT limitations such as shift sensitive, poor directionality.

      Figure 5(c) Fused Image using DWT (sym5)

      Figure 5(e) Reference image

      Table 1. Comparison of image quality metrics for the different image fusion techniques

      Fusion Techniques Performance Metrics	DWT [10]		DTCWT
      	Db5	Sym5
      MSE	12.8520	13.6680	12.495
      PSNR	37.0403	36.7737	37.1601
      NCC	0.99919	0.99912	0.99923

      From Table 1, it is clear that, the performance of the DTCWT is better than that of DWT based fusion technique. In the DWT fusion, technique db5 wavelet filters suits better than the sym5 wavelet filters and better performance wavelet will vary image to image. It is not necessary that, particular wavelet filters suits to all the required applications. It depends on the correlation between the filter coefficients and intensity of the image.

   6. CONCLUSION

In this work, Image fusion techniques based on Discrete Wavelet Transform, and Dual Tree Complex Wavelet Transform are studied. This paper emphasizes the Laplacian Pyramidal decomposition and Spatial Frequency parameter based fusion rules in DWT and DTCWT techniques for the improved results. The performance of the fusion techniques is judged by the image quality metrics like PSNR, MSE and NCC.

Figure 5(d) Fused Image using DTCWT

Figure 5(e) shows the reference image, for the calculation of image performance metrics like PSNR, MSE and NCC. The image quality (performance) metrics MSE, PSNR and NCC are computed for the fused image with respect to the reference image shown in Figure 5(e), and the performance metrics are listed in the Table 1

In summary, the Dual Tree Complex Wavelet Transform with Laplacian Pyramidal decomposition and spatial frequency fusion rules, is found to be more efficient than DWT based fusion technique, because of the limitation of DWT. Further, as the number of decomposition levels increases, the fused image quality also increases, but the number of computation for spatial frequency computation increases, and hence the running time of the program. However, the running time is not proportionate with the number of decomposition levels.

REFERENCES

1. N.G. Kingsbury. The dual-tree complex wavelet transform with improved orthogonality and symmetry propertiesIEEE International Conference on Image Processing,Sep. 2000, pp. 375378,

2. S. Mallat, A Theory for Multiresolution Signal Decomposition: The Wavelet Representation, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 7, pp. 674693, 1989.

3. T. Pu and G. Ni, Contrast-based Image Fusion using the Discrete Wavelet Transform, Optical Engineering, vol. 39no. 8, pp. 20752082, 2000.

4. Eskicioglu, A.M., Fisher, P.S, Image quality measure and their performance, IEEE Trans. Comm., Vol. 43, No.12, 2959-2965

5. B. Ajazzi, L. Alparone, S. Baronti, and R. Carla, Assessment of pyramid-based multisensor image data fusion, Image and Signal Processing for Remote Sensing, vol. 4, pp. 237248, 1998

6. F. Sadjadi, Comparative image fusion analysis, in Proc. IEEE Conf.

   Comput. Vision PatternRecogn., San Diego, CA,, vol. 3, Jun. 2005

7. Y. Zhang, Understanding image fusion, Photogramm. Eng. Remote Sens., vol. 70, no. 6,pp. 657661, Jun. 2004.

8. Krista Amolins, Yun Zhang, and Peter Dare, Wavelet based image fusion techniquesAnintroduction, review and comparison, ISPRS Journal of Photogrammetric and Remote Sensing, Vol. 62, pp. 249-263, 2007.

9. L.J. Chipman, T.M. Orr, and L.N. Lewis. Wavelets and image fusion.IEEE Transactionson Image Processing, 3:248251, 1995.

10. N. Raj, M. Chandrasekhar, Enhanced Image Fusion Algorithm Using Laplacian Pyramid and Spatial frequency Based Wavelet Algorithm, International Journal of Soft Computing and Engineering (IJSCE), ISSN: 2231-2807, Vol. 2, no.3, pp. 298-303, Dec-2014