Multi-Modality Medical Image Fusion – A Survey

Multi-Modality Medical Image Fusion – A Survey

1 Bhavana. V 2 Dr. Krishnappa H. K

VI Semester M.Tech (QIP) Associate Professor Dept. of CSE Dept. of CSE

R V College of Engineering R V College of Engineering Bangalore Bangalore

Abstract – Medical imaging modalities such as magnetic resonance imaging (MRI), computerized tomography (CT), positron emission tomography (PET), have been developed and widely used for clinical diagnosis. In brain medical imaging, MR image provides high-resolution anatomical information in gray intensity, while PET image reveals the biochemical changes in color without anatomical information. These two types of images contain important information that a brain disease can be diagnosed accurately and effectively. Thus, fusing two different medical images into a single image with both anatomical structural and spectral information is highly desired. This paper discusses various fusion techniques that can be used for fusing multi-modality medical images.

Keywords: Multimodal medical images, image fusion, medical imaging.

1. INTRODUCTION

   In medical field, doctors (radiologists) require high spatial and high spectral information in a single image for the purposes like researches, monitoring, accurate diseases diagnosing and for treatment process. This type of information cannot be obtained using single modality images since, Computed Tomography (CT) image which are most popular for showing bone structures and lacks in providing information about the tissues at the same time, Magnetic Resonance Imaging (MRI) image which provides soft tissues information and lacks in boundary information, Positron Emission Tomography (PET) image which provides clear information in blood flow but lacks in boundary information and so on. Thus, every single modality images has their own drawbacks in providing needed information because each image is captured with different radiation power. To resolve this, complementary information from multiple modalities is required. In this situation, fusion is a technique used to combine multi- modality medical images such as CT, MRI, PET etc. Image fusion is a technique of combining appropriate information from two or more images into a single fused image where the resulting image can provide more inventive information than any of the input images[1]. These fused image information are used in many applications such as Aerial and Satellite imaging, Medical imaging, Robot vision, Multi-focus image fusion etc.

2. LITERATURE REVIEW

T.Zaveri et al. (2009) [1] explained that the image fusion is a process of combining multiple input images of the same scene into a single fused image, which contains important information and obtain the important features from each of the original images and makes it more suitable for human and machine perception. A novel region based image fusion method is explained in this paper which shows that region based image fusion algorithm performs better than pixel based fusion method. Pixel level image fusion methods are affected by blurring effect which directly affect on the contrast of the image. Therefore the paper describes region based method which is less sensitive to noise, better contrast and less affected by mis- registration. The large number of registered images is applied by the proposed algorithm and results are compared using standard reference. The proposed method performs well compared to other methods because this method is less sensitive to noise.

Yijian Pei et al. (2010) [2] after studying the principles and characteristics of discrete wavelet framework, explained an improved discrete wavelet framework based image fusion algorithm. The improvement is considered of the high frequency sub band image region characteristic. The wavelet transform based algorithm can obtain less noise than weighted average algorithm. The useful information of each source image is retrieved from the multi sensor, if the algorithm is synthesized effectively. The multi focus image fusion results are more accurate and reliable. Therefore this method can result in less data size, more efficient target detection and situation estimation for observers. The proposed method efficiently fuses the features and information of each image and hence the feasibility of wavelet in image fusion is also verified. The multi focus image fusion experiments and medical image fusion experiments can verify that this proposed algorithm has the effectiveness in the image fusion. This paper illustrated the quality assessment of the image fusion and quantitatively analyze the performance of algorithms. The proposed method synthetically corrects the quality by the subjective assessment method and the objective assessment method. The assessment result shows that the proposed algorithm can fuse the images information in better performance.

Patil et al. (2011) [3] has focused on image fusion algorithm using hierarchical PCA. This paper also describes that the image fusion is a process of integrating two or more images of the same scene to get the more informative image. In this paper, the author proposed an image fusion algorithm by combining pyramid and PCA techniques and carryout the quality analysis of proposed fusion algorithm without reference image which can be used for feature extraction, dimension reduction and image fusion.

S. Daneshvar et al. (2011) [4] proposed an algorithm that integrates the advantages of both IHS and RIM fusion methods to improve the functional and spatial information content. Visual and statistical analyses show that the proposed algorithm significantly improves the fusion quality in terms of entropy, mutual information, discrepancy, and average gradient compared to the fusion methods including IHS, Brovey, discrete wavelet transform (DWT), a-trous wavelet and RIM. Image fusion has become a widely used tool for increasing the interpretation quality of images in medical applications. The acquired data might exhibit either good functional characteristic (such as PET) or high spatial resolution (such as MRI). The MRI image shows the brain tissue anatomy and contains no functional information. The PET image indicates the brain function and has a low spatial resolution. Hence, the image fusion task is carried out to enhance the spatial resolution of the functional images by combining them with a high- resolution anatomic image. A perfect fusion process preserves the original functional characteristics and adds spatial characteristics to the image with no spatial distortion. The intensity-hue-saturation (IHS) algorithm and the retina-inspired model (RIM) fusion technique preserves more spatial feature and more functional information content respectively.

Phen-Lan Lin et al. (2011) [5] proposed two fusion methods, IHS&LG+ and IHS&LG++, based on IHS and log-Gabor wavelet for fusing PET and MRI images by choosing suitable decomposition scale and orientation for different regions of images in the first method, and refining the fused intensity of the first method to further reduce color distortion and enforce the anatomical structure in the second method. This methods use the hue angle of each pixel in PET image to divide both PET and MRI images into regions of high and low activity. The fused intensity of each region is obtained by inverse log-Gabor transforming of high frequency coefficients of MRI intensity and low frequency coefficients of PET intensity-component. The experiments are performed on three sets of normal axial, normal coronal, and Alzheimers disease which demonstrate that all three images fused by IHS&LG+ are with less color distortionand about the same structural information as the images fused by IHS&RIM, and all three images fused by IHS&LG++ are with both color and anatomical structural information closest to PET and MRI images both visually and quantitatively.

Prakash et al. (2012) [6] explained that the image fusion is basically a process where multiple images are combined to form a discrete resulting fused image. This fused image is more active as compared to its original input images. The fusion technique in medical images is useful for ingenious disease diagnosis purpose. This paper illustrated different multi-modality medical image fusion techniques and their results are assessed with various quantitative metrics. CT and MRI-T2 are taken as input and then the fusion techniques are applied to the input images such as Mamdani type minimum sum mean of maximum (MIN- SUM-MOM) and Redundancy Discrete Wavelet Transform (RDWT) and the resultant fused image is analyzed with quantitative metrics such as Overall Cross Entropy(OCE),Peak Signal to Noise Ratio (PSNR), Signal to Noise Ratio(SNR), Structural Similarity Index(SSIM), Mutual Information(MI).In this paper the authors proved that Mamdani type MIN-SUM-MOM is more productive than RDWT.

Maruturi Haribabu et al. (2012) [7] proposed a new approach for PET- MRI image fusion by using the wavelet and spatial frequency method. In the proposed method the influence of image imbalance is eliminated and blurred the phenomenon of fusing image, improved the clarity and provided more reference information for doctors. The result shows that the performance of the proposed method is superior to the traditional algorithm based on PCA in terms of good visual & quantitative analysis fusion results.

Desale et al. (2013) [8], explained the Formulation, Process Flow Diagrams and algorithms of PCA (Principal Component Analysis), DCT (Discrete Cosine Transform) and DWT based image fusion technique. The PCA & DCT are conventional fusion techniques with many drawbacks, whereas DWT based techniques are better as they provides better results for image fusion. In this paper, the authors proposed two algorithms based on DWT called pixel averaging & maximum pixel replacement approach and their results are compared accordingly.

Phen-Lan Lin et al.(2014) [9], proposed a new method based on PET and MR brain image fusion based on wavelet transform for low and high activity brain image regions respectively. The proposed method can generate very good fusion result by adjusting the anatomical structural information in the gray matter (GM) area, and then patching the spectral information in the white matter (WM) area after the wavelet decomposition and gray-level fusion. A novel adjustment for the pixel intensity in the non-white matter area of high-activity region in the gray- level fused image will bring more anatomical structural information into the final color fused image. Spectral information patching in the white matter area of high- activity region will preserve more color information from PET image for the white-matter area. The fusion results are compared based on the performance metrics – spectral discrepancy (SD) and average gradient (AG).

COMPARISON OF VARIOUS FUSION TECHNIQUES

CONCLUSION

This paper explains the different related works based on fusion techniques used for multimodal medical images. The various fusion techniques, their advantages and disadvantages are discussed. The comparative analysis of image fusion techniques allows in selecting the best fusion method and therefore one can obtain better visualization of the fused image.

REFERENCES

1. Tanish Zaveri, Mukesh Zaveri, A Novel Region Based Multifocus Image Fusion Method, International Conference on Digital Image Processing, 978-0-

   7695-3565-4/09, 2009 IEEE DOI 10.1109/ICDIP.2009.27.

2. Yijian Pei, Jiang Yu, Huayu Zhou, Guanghui Cai , The Improved Wavelet

   Transform Based Image Fusion Algorithm and the Quality Assessment, 2010

   3rd International Congress on Image and Signal Processing (CISP2010),978-1-424 4-6516-3/10,2010IEEE.

3. Patil, Ujwala and Uma Mudengudi. "Image fusion using hierarchical PCA." In image Information Processing (ICIIP), 2011 International Conference on, pp. 1-6. IEEE, 2011.

4. S.Daneshvar and H.Ghassemian, MRI and PET image fusion by combining IHS and retina-inspired models ,In Information Fusion, vol.11, pp.114-123, 2010.

5. Phen-Lan Lin, Po- Whei Huang, Cheng- I Chen, Tsung-Ta Tsai and Chin-han Chan, Brain Medical Image Fusion Based On HIS And LOG-GABOR with suitable decomposition scale and orientation for different regions, ADIS International Conference Computer Graphics, Visualization, Computer Vision and Image Processing 2011, ISBN: 978-972-8919-48-9,2011 IADIS.

6. Prakash, Chandra, S. Rajkumar, and

   P. V. S. S. R. Mouli. "Medical image fusion based on redundancy DWT andMamdani-type min-sum mean-of-max techniques with quantitative analysis." In Recent Advances in Computing and Software Systems (RACSS), 2012 International Conference on pp. 54-59. IEEE, 2012.

7. Maruturi Haribabu, CH Hima Bindu, Dr. K. Satya Prasad, Multimodal Image Fusion of MRI-PET Using Wavelet Transform, 2012 International Conference on Advances in Mobile Network, Communications and its Applications, 22IEEE,DOI10.1109/MNCApps.2012.33

8. Desale, Rajenda Pandit, and Sarita

   V. Verma. Study and analysis of PCA, DCT & DWT based image fusion techniques." In Signal Processing Image Processing & Pattern Recognition(ICSIPR), 2013 International Conference on pp. 66-69. IEEE, 2013.

9. Po-Whei Huang, Cheng- I Chen, Phen- Lan Lin, Ping Chen, Li- Pin Hsu, PET and MRI Brain Image Fusion Using Wavelet Transform with Information Adjustment and Spectral Information Patching, 2014 IEEE International Symposium on Bioelectronics and Bioinformatics (IEEE) ISBB 2014), 978-1-4799-2770-8/14, 2014 IEEE.