Directive Contrast Based Multimodal Medical Image Fusion with Non-Sub sampled Contour Let Transform

Directive Contrast Based Multimodal Medical Image Fusion with Non-Sub sampled Contour Let Transform

G. Selvakumari

PG Student, ECE,

Jayaram College of Engineering and Technology, Trichy, India.

R. Aravindh

Assistant Professor, ECE,

Jayaram College of Engineering and Technology, Trichy, India.

Abstract- This paper presents the multimodal medical image fusion technique based on discrete non-subsampled contourlet transform and pixel level fusion rule. The fusion is to minimize different errors between the fused image and the input images. In medical diagnosis, the edges and outlines of the interested objects is more important than other information. Therefore, how to preserve the edge-like features is worthy of investigating for medical image fusion. As we know, the image with higher contrast contains more edge-like features. In term of this view, the paper proposed a new medical image fusion scheme based on discrete contourlet transformation, which is useful to provide more details about edges at curves. It is used to improve the edge information of fused image by reducing the distortion. This transformation will decompose the image into finer and coarser details and finest details will be decomposed into different resolution in different orientation. The pixel and decision level fusion rule will be applied selected for low frequency and high frequency and in these rule we are following image averaging, phase congruency and directive contrast. The fused contourlet coefficients are reconstructed by inverse NS contourlet transformation. The visual experiments and quantitative assessments demonstrate the effectiveness of this method compared to present image fusion schemes, especially for medical diagnosis. For medical diagnosis, doctors usually observe the images manually and fuse them in the mind. The goal of image fusion is to obtain useful complementary information from CT/MRI multimodality images. By this method we can get more complementary information and also satisfactory Entropy, Better correlation coefficient, PSNR (Peak- Signal-to-Noise Ratio) and less MSE (Mean square error).

Key Words—- Multimodal medical image fusion, non- subsampled contourlet transform, phases congruency.

I. INTRODUCTION

Recently, medical imaging has more increasing attention due to its crucial role in health care. However, some different types of imaging techniques such as X-ray, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Magnetic Resonance Angiography (MRA), etc., provide limited information where some information is common, and some are unique. For example, X-ray and Computed tomography (CT) can provide dense structures like bones with minimum distortion, but it cannot detect any physiological changes. Similarly, MRI images can provide normal and pathological soft tissue. Hence, the anatomical and functional

properties of medical images are must to be combined for a compendious view. So for this purpose, the multimodal medical image fusion has been identified as a promising solution which aims to integrate the information from multiple modality images so as to obtain a more accurate and complete description of the same object. Multimodal medical image fusion helps in diagnosing diseases, as well as it also reduces the storage cost by reducing the storage to a single fused image instead of storing multiple-source images [1].

So far, many research works has been made on image fusion technique with various techniques respected to multimodal medical image fusion. According to merging stage, these techniques have been categorized into three categories. These include pixel level, feature level and decision level fusion. Fusion in medical image usually employs the pixel level fusion due to the advantage of containing the original measured quantities, easy implementation and efficient computation. Hence, in this paper, we concentrate on pixel level fusion, and the terms fusion or image fusion are mostly used for pixel level fusion. The pixel level fusion is based on principal component analysis (PCA), independent component analysis (ICA), contrast pyramid (CP), gradient pyramid (GP) filtering, etc. The image features are sensitive to the human visual system which exists in different levels of scales. Hence, these are not highly suitable for medical image fusion. In the recent years, with the development of multiscale decomposition, wavelet transform has been identified as the ideal method for fusion of image. However, it is stated that wavelet decomposition is good only at isolated discontinuities, but not good at edges and textured region. Also, it captures only very limited directional information along vertical, horizontal and diagonal direction [2]. These issues are overcome in a recent multiscale decomposition called as contourlet transform, and its non- subsampled version. Contourlet is a 2-D sparse representation where sparse expansion is expressed by contour segments for 2-D signals like images. So, as a result, this transform can capture 2-D geometrical structures in visual information in much more effective manner other than traditional multiscale methods.

Here, a novel fusion framework for multimodal medical images based on non-subsampled contourlet transform is proposed. The main idea is to perform NSCT on the medical images followed by the fusion of low and high frequency

coefficients. The phase congruency and directive contrast feature for low and high-frequency coefficients are unified as the fusion rules. The phase congruency provides a contrast and brightness-invariant representation of low-frequency coefficients whereas directive contrast efficiently determines the frequency coefficients from the clear parts in the high frequency. The combinations of these two can preserve more details in source images and further improve the quality of fused image. The efficiency of the proposed framework is carried out by the extensive fusion experiments on different multimodal CT/MRI dataset.

The salient contributions of the proposed framework over many existing methods can be summarized as follows.

• This paper proposes a new image fusion framework for multimodal medical images, which relies on the NSCT.

• Two different fusion rules are proposed for combining low frequency and high-frequency coefficients.

• For fusing the low-frequency coefficients, the phase congruency method is used. The main benefit of phase congruency is that it selects and combines contrast and brightness-invariant representation contained in the low- frequency coefficients.

• In the same way, a new definition of directive contrast in NSCT domain is proposed and used to combine high frequency coefficients. Using directive contrast, the most prominent texture and edge information are selected from high-frequency coefficients and combined in the fused images.

• The definition of directive contrast is consolidated by incorporating a visual constant to the SML based definition of directive contrast which provides a richer representation of the image.

The rest of the paper is organized as follows .NSCT and phase congruency is described in Section 2 followed by the proposed multimodal medical image fusion framework in Section 3. In Section 4 Experimental results and discussions are given and the concluding remarks are described in Section 5.

combining the directional fan filter banks. NSDFB allows the direction decomposition with stages in high-frequency images from NSP at each scale and produces directional sub-images with the same size as the source image. Therefore, the NSDFB offers the NSCT with the multi-direction property and provides us with more precise directional details information. A four channel NSDFB constructed with two-channel filter banks is illustrated in Fig.2.

Fig. 1. Three-stage non-subsampled pyramid decomposition.

II .PRELIMINARIES

This section describes the description of concepts on which the proposed framework is based.

1. Non-Subsampled Contourlet Transform (NSCT)

   NSCT, based on the theory of Contourlet Transform, is a kind of multi-scale and multi-direction computation framework of the discrete images [3]. It can be divided into two stages including non-subsampled pyramid (NSP) and non- subsampled directional filter bank (NSDFB). The former stage ensures the multiscale property by using two-channel non- subsampled filter bank, and one low-frequency image and one high-frequency image can be produced at each NSP decomposition level. The subsequent NSP decomposition stages are carried out to decompose the low-frequency component available iteratively to capture the singularities in the image. As a result, NSP can result in k+1 sub-image, which consists of one low- and k high-frequency images having the same size as the source image where k denotes the number of decomposition levels. Fig.1 gives the NSP decomposition with k=3 levels. The NSDFB is two-channel non-subsampled filter banks which are constructed by

   Fig. 2. Four-channel non-subsampled directional filter bank.

2. Phase Congruency

Phase congruency is a measure of feature perception in the images which provides an illumination and contrast invariant feature extraction method. This approach is based on the Local Energy Model, which postulates that significant features can be found at points in an image where the Fourier components are maximally in phase. Furthermore, the angle at which phase congruency occurs signifies the feature type. The phase congruency approach to feature perception has been used for feature detection. First, logarithmic Gabor filter banks at different discrete orientations are applied to the image and the local amplitude and phase at a point are obtained.

The main properties, which acted as the motivation to use phase congruency for multimodal fusion, are as follows.

• The phase congruency is invariant to different pixel intensity mappings. The images captured with different modalities have significantly different pixel mappings, even if the object is

  same. Therefore, a feature that is free from pixel mapping must be preferred.

• The phase congruency feature is invariant to illumination and contrast changes. The capturing environment of different modalities varies and resulted in the change of illumination and contrast. Therefore, multimodal fusion can be benefitted by an illumination and contrast invariant feature.

• The edges and corners in the images are identified by collecting frequency components of the image that are in phase. As we know, phase congruency gives the Fourier components that are maximally in phase. Therefore, phase congruency provides the improved localization of the image features, which lead to efficient fusion.

1. PROPOSED MULTIMODAL IMAGE FUSION FRAMEWORK

   In this section, we have discussed some of the motivating factors in the design of our approach to multimodal medical image fusion. The proposed framework realizes on the directive contrast and phase congruency in NSCT domain, which takes a pair of source image denoted by A and B to generate a composite image F. The basic condition in the proposed framework is that all the source images must be registered in order to align the corresponding pixels. The block diagram of the proposed framework is depicted in Fig.3 but before describing it, the definition of directive contrast is first described.

   Fig. 3. Block diagram of proposed multimodal medical image fusion framework.

   1. Directive Contrast in NSCT Domain

      The contrast feature measures the difference of the intensity value at some pixel from the neighboring pixels. The human visual system is highly sensitive to the intensity contrast rather than the intensity value itself. Generally, the same intensity value looks like a different intensity value depending on intensity values of neighboring pixels. Therefore, local contrast is developed and is defined as [4]

      local background. Generally, LB is regarded as local low- frequency and hence LB=LH is treated as local high-frequency. This definition is further extended as directive contrast for multimodal image fusion [7]. These contrast extensions take high-frequency as the pixel value in multiresolution domain. However, considering single pixel is insufficient to determine whether the pixels are from clear parts or not. Therefore, the directive contrast is integrated with the sum-modified- Laplacian to get more accurate salient features. In general, the larger absolute values of high-frequency coefficients correspond to the sharper brightness in the image and lead to the salient features such as edges, lines, region boundaries, and so on. However, these are very sensitive to the noise and therefore, the noise will be taken as the useful information and misinterpret the actual information in the fused images. Hence, the sum-modified-Laplacian is integrated with the directive contrast in NSCT domain to produce accurate salient features

      In order to accommodate for possible variations in the size of texture elements, a variable spacing (step) between the pixels is used to compute partial derivatives to obtain SML and is always equal to 1. Further, the relationship between the contrast sensitivity threshold and background intensity is nonlinear, which makes the human visual system highly sensitive to contrast variation [6]. Hence, the above integration must be improved to provide better details by exploiting visibility of low-frequency coefficients in the above- mentioned definition. The proposed definition of directive contrast, defined by (7), not only extract more useful features from high-frequency coefficients but also effectively deflect noise to be transferred from high-frequency coefficients to fused coefficients.

   2. Proposed Fusion Framework

      Considering, two perfectly registered source images A and B the proposed image fusion approach consists of the following steps:

      1. Perform -level NSCT on the source images to obtain one low-frequency and a series of high-frequency sub-images at each level and direction .

      2. Fusion of Low-frequency Sub-images: The coefficients in the low-frequency sub-images represent the approximation component of the source images. The simplest way is to use the conventional averaging methods to produce the composite bands. However, it cannot give the fused low-frequency component of high quality for medical image because it leads to the reduced contrast in the fused images. Therefore, a new criterion is proposed here based on the phase congruency. The complete process is described as follows.

         • First, the features are extracted from low-frequency sub-images using the phase congruency extractor.

         • Fuse the low-frequency sub-images.

           = =

           (1)

           where L is the local luminance and LB is the luminance of the

      3. Fusion of High-frequency Sub-images: The coefficients in the high-frequency sub-images usually include details component of the source image. It is noteworthy that the noise is also related to high-

         frequencies and may cause miscalculation of sharpness value and therefore affect the fusion performance. Therefore, a new criterion is proposed

         here based on directive contrast. The whole process is described as follows.

         • First, the directive contrast for NSCT high-frequency sub-images at each scale and orientation.

         • Fuse the high-frequency sub-images.

      4. Perform level inverse NSCT on the fused low- frequecy and high-frequency sub-images, to get the fused image.

2. RESULTS AND DISCUSSIONS

   Some general requirements for fusion algorithm are: (1) it should be able to extract complimentary features from input images, (2) it must not introduce artifacts or inconsistencies according to Human Visual System and (3) it should be robust and reliable. Generally, these can be evaluated subjectively or objectively. The former relies on human visual characteristics and the specialized knowledge of the observer, hence vague, time-consuming and poor-repeatable but are typically accurate if performed correctly. The other one is relatively formal and easily realized by the computer algorithms, which generally evaluate the similarity between the fused and source images. However, selecting a proper consistent criterion with the subjective assessment of the image quality is rigorous. Hence, there is a need to create an evaluation system. Therefore, first an evaluation index system is established to evaluate the proposed fusion algorithm. These indices are determined according to the statistical parameters.

   1. Evaluation Index System

      1. Normalized Mutual Information: Mutual information (MI) is a quantitative measure of the mutual dependence of two variables.

         It usually shows measurement of the information shared by two images. Mathematically, MI between two discrete random variables U and V is defined as,

         ()

         , = (, )2 (,) (2) where (, ) is the joint probability distribution function U and V. Based on the above definition, the quality of the fused image with respect to input images and can be expressed as

   2. Experiments on CT/MRI Image Fusion

      To evaluate the performance of the proposed image fusion approach, four different datasets of human body parts are considered (see Fig. 4). The images in Figs. 4(a),(c),(e),(g) and (b),(d),(f),(h) are CT and MRI. The corresponding pixels of two input images have been perfectly co-aligned. All images have the same size of 256×256 pixels, with 256-level gray scale. The proposed medical fusion technique is applied to these image sets. It can be seen that due to various imaging principle and environment, the source images with different modality contain complementary information. For implementing NSCT, maximally flat filters and diamond max- flat filters are used as pyramidal and directional filters respectively.

      Fig-4: Multimodal medical image data sets: (a), (c), (e), (g) CT image and (b), (d), (f), (h) MRI image

      Among multiresolution based algorithms, the algorithms based on NSCT perform better. This is due to the fact that NSCT is a multi-scale geometric analysis tool which utilizes the geometric regularity in the image and provides an asymptotic optimal representation in the terms of better localization, multi-direction and shift invariance. This is also justified by the fact that shift-invariant decomposition improves the quality of the fused image around edges. If the NSCT based methods have been compared then it can be observed that the performance of the proposed method is

      = 2 (,)

      +()

      (,)

      +

      +()

      (3)

      better than existing NSCT based methods [5]. This algorithm uses a directional vector, obtained from high frequency sub-

      1. Structural Similarity based Metric: Structural similarity (SSIM) is designed by modeling any image distortion as the combination of loss of correlation, radiometric and contrast distortion. Mathematically, SSIM between two variables U and V is defined as

         bands, to fuse low-frequency sub-bands. This directional vector essentially defines the clarity factor and is used to collect pixels from blur and clear regions. This algorithm performs somewhat well in the case of multifocus images but the performance degraded when it is applied to the medical images. This is because this algorithm is not able to utilize

         2 2 2 2

         , = 2 2

         + +

         (4)

         prominent information present in the low-frequency efficiently and results in the poor quality. The performance of the

      2. Edge Based Similarity Measure: The edge based similarity

      measure gives the similarity between the edges transferred in

      the fusion process. Mathematically, is defined

      as,

      proposed method provides the good quality fused images compared to others. Fig.5 shows the result of multimodal medical image fusion.

      +

      = =1 =1 , , , ,

      (5)

      +

      =1

      =1

      , ,

      Fig5. The multimodal medical image fusion result of fusion algorithm

      (a) fusion result of dataset 1, (b) fusion result of dataset 3, (c)fusion result of dataset 2, (d) fusion result of dataset 4.

   3. Performance Analysis

   We can analysis the performance by measuring the metrics such as quality, PSNR and entropy. The performance analysis for the four different datasets is given in the following table.

   TABLE I. EVALUATION INDICES FOR FUSED MEDICAL IMAGES

   Image Datasets	Performance metrics
   	Quality	SSIM	PSNR	Entropy	Correlation
   Dataset 1	87.39	0.2878	11.4044	1.23	0.9581
   Dataset 2	99.51	0.5424	11.5469	2.66	1
   Dataset 3	97.36	0.0177	10.1018	1.31	0.8671
   Dataset 4	99.58	0.5186	12.5101	2.73	1

3. CONCLUSION

In this paper, a novel image fusion framework is proposed for multi-modal medical images, which is based on non- subsampled contourlet transform and directive contrast. For fusion, two different rules are used by which more information can be preserved in the fused image with improved quality. The low frequency bands are fused by considering phase congruency whereas directive contrast is adopted as the fusion measurement for high-frequency bands. In our experiment, four groups of CT/MRI images are fused using conventional fusion algorithms. The visual and statistical comparisons demonstrate that the proposed algorithm can enhance the details of the fused image, and can improve the visual effect.

ACKNOWLEDGMENT

First and foremost we thank God, the almighty who stands behind and strengthens me to complete the work successfully. We would like to express my sincere respect and gratitude towards our HOD, Mrs.Geetha.M, M.E., Her wide knowledge; serious research attitude and enthusiasm in work deeply impressed us and taught what a true scientific research should be. We are very thankful for the support she extended to us and the freedom to express our views. Words are inadequate to express the gratitude to our beloved parents and friends for their excellent and never ending co-operation.

REFERENCES

1. G.Bhatnagar, Q.M. J.Wu, and Z. Liu, Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain, in IEEE Transactions on Multimedia, vol,15, no.5, pp.1014-1024, 2013.

2. F. E. Ali, I. M. El-Dokany, A. A. Saad, and F. E. Abd El-Samie, Curvelet fusion of MR and CT images, Progr. Electromagn. Res. C, vol. 3, pp. 215224, 2008.

3. A. L. da Cunha, J. Zhou, and M. N. Do, The non-subsampled contourlet transform: Theory, design, and applications, IEEE Trans. Image Process., vol. 15, no. 10, pp. 30893101, Oct. 2006.

4. A. Toet, Hierarchical image fusion, Mach. Vision ppl., vol. 3, no. 1, pp. 111, 1990.

5. Q. Zhang and B. L. Guo, Multifocus image fusion using the non- subsampled contourlet transform, Signal Process., vol. 89, no. 7, pp. 13341346, 2009.

6. A. B.Watson, Efficiency of a model human image code, J. Opt. Soc. Amer. A, vol. 4, no. 12, pp. 24012417, 1987.

7. G. Bhatnagar and B. Raman, A new image fusion technique based on directive contrast, Electron. Lett. Comput. Vision Image Anal., vol. 8, no. 2, pp. 1838, 2009.

8. V. S. Petrovic and C. S. Xydeas, Gradient-based multiresolution image fusion, IEEE Trans. Image Process., vol. 13, no. 2, pp.228237, Feb. 2004.

AUTHORS BIOGRAPHY

G.Selvakumari received her B.E. degree in Electronics and Communication Engineering from Sethu Institute of Technology, Virudhunagar, Anna University, Chennai, India, in 2012 and M.E degree in Communication Systems from Jayaram College of Engineering & Technology, Trichy, Anna University, Chennai, India, in 2014. Her research area of interest is Image Processing and Linear Integrated Circuits.

R.Aravindh received his B.E. degree in Electronics and Communication Engineering from the Jayaram College of Engineering and Technology, Trichy, Anna University, Chennai, India, in 2008. He Received

M.E. degree in Electronics and Communication Engineering (Communication Systems) in Jayaram College of Engineering and Technology, Trichy, Anna University of technology, Trichy, India in 2011.He is currently working as an Assistant Professor in the Department of Electronics and Communication Engineering in Jayaram College of Engineering and Technology, Trichy. His research interest includes Image Processing, wireless communication (Wi-Fi, Wi-Max), Mobile Ad hoc networks, Sensor Networks, Neural Networks and fuzzy logic, Communication networks.