Positioning Lesion from Breast MRI and Mammogram using Registration Method

Positioning Lesion from Breast MRI and Mammogram using Registration Method

Rugma M.*, Malu G., C. Gopakumar, Elizabeth Sherly¶

*PG Scholar, Dept. of CE, College of Engineering, Chengannur. Alapuzha, India

Research Scholar, IIITM-K, Technopark, Trivandrum, India

Associate Professor, Dept. of ECE, College of Engineering, Karunagappally. Kollam, India

¶Professor, IIITM-K, Techopark, Trivandrum, India

Abstract The mammogram and MRI used for detection of cancerous cells in breast, requires overlaying the approach to get the best out of two or a combined effort. However, the varying intensity of breast in both modalities make the automatic segmentation of such lesions extremely challenging as both the modalities are different. This paper presents a new approach for automatic registration of mammograms and MR images. The registration of images taken from MRI and mammography is carried out to locate the position of lesion present in breast followed by noise removal, edge enhancement and morphological operations. The chest wall removal is done to extract the breast area, for obtaining the mass area and micro-calcifications to locate its position. The locations of lesions in both images are extracted out using region growing algorithm and thresholding method. Registering the images provide the most accurate location of masses and micro-calcification that is missed in any of the modalities, into single image. The results show that the method gives fast and accurate registration that can be effectively used for detecting masses and micro-calcifications with high level of accuracy. The accuracy obtained in the proposed method is about 87%.

Keywordsimage pre-processing, chest wall removal, nipple localization, segmentation, registration.

1. INTRODUCTION

   Breast cancer continues to be a significant public health problem in the world. Early detection is the better method for improving breast cancer prognosis. Various researches were carried out for the automatic detection of breast cancer independently, but work on combining both MRI and mammogram is very less. In this paper we propose a technique for locating the cancerous lesions present in breast from multimodality images such as: Magnetic Resonance Imaging (MRI) and mammogram images using image registration [1]. The goal is to attain the accurate position of breast lesion by combining both the multi-modal images. The processes involved are image pre-processing, chest wall removal, nipple localization, segmentation and registration. The proposed technique has the potential to support clinical diagnosis, which helps medical practitioner in correctly identifying the position of breast lesion.

   Mammography is currently the primary method of early detection. But recent research has shown that many cases

   missed by mammography can be detected in Breast Magnetic Resonance Imaging (MRI). In proposed system, the Medio- Lateral Oblique (MLO) view of (X-ray) mammograms [2] is considered for registering with the MR images. Since, the mammogram images considered for registration is oblique i.e., MLO mammograms, the MR slices for the proposed system should be of side to side lengthwise (sagitall) images. These images are comparable and used in registration to locate the breast lesion. MRI provides good tissue specificity via Gadolinium Diethylene Triamine Penta-Acetic Acid (Gd- DTPA) of contrast enhancement, whilst mammography has spatial resolution and image formation capable of providing information about curvilinear structures, tumour spiculation and micro calcifications.

   The objective of this paper is to register the MLO mammograms to equivalent projections of a contrast enhanced MR slices to enable feature correspondence to be made between modalities. Here, the MR images are obtained from Signa Hdxt Magnetic Resonance Scanner and the mammograms are obtained from selenia dimensions. The image that is obtained through diagnosis is in DICOM (Digital Imaging and Communications in Medicine) format which is three dimensional images. The MLO images considered consist of chest wall which is comparable with the chest walls in MR images. The chest wall removal plays an important part in lesion localization. Otherwise, the presence of nodes at the walls of the breast gives a false identification of lesion making segmentation more complex. Registering both the images provides high quality resolution images and the details lost in either of the image during diagnosis can be retained.

   The overall organization of the paper is as described below. The general problem in finding the breast lesion from a single image is mentioned in section II. Section III describes the proposed methodology for registration and in each subsection it has been explained in details. Section IV demonstrates some simulation results and their performance evaluation. Finally conclusions are presented in section V which also narrates the advantages of the proposed registration technique.

2. PROBLEM FORMULATION Mammography [8] is the most popular and cost effective

   image diagnosis for breast cancer detection. However, it is not

   possible to record all types of lesion using mammogram alone. Sometimes the physician has to rely on some other imaging techniques along with mammography for better identification of lesion. Mammography is good in micro calcification and tumor specifications, but tissues with hard nature are not prominent in it. In such cases, MRI is preferred due to its non- invasive nature and tissue specificity. Though, MRI is bit expensive, it provides good details on masses missed by mammography. In the case of large amount of data, relaying on both images for lesion prediction is not appropriate for operator assisted method. In such cases, a co-registration process to combine both the images provides much easiness, but is challenging mainly because of the multimodality of two images. Here an attempt is made to construct co-registration of two multimodality images to extract more information, so that diagnosis will be more accurate and easy.

   In previous work, most of the authors considered the carnio caudal (CC) view of mammogram for registration with MRI. The CC mammogram gives an ellipsoidal shape to compare it with the MR images. But the complete breast capturing in such images is still a daunting problem. The patients with low breast volume will only be able to produce the complete structure of breast during mammography, while patients with high breast volume fails doing so. As a result, registering such images becomes tedious and there is a high chance for missing relevant information.

   To overcome this situation, the MLO mammograms [2] are considered for registration with MR images. The presence of chest wall in MLO images indicates complete capture of breast during mammography. Since, the MLO views of mammogram and sagitall images of MR are comparable; these views can be used for registration activity. The chest wall removal in proposed method helps to improve the flawless working of the system.

3. PROPOSED METHOD

   The proposed system aims to develop an efficient method to register the multi-modal images for detecting the masses and micro-calcifications present in breast. The system is designed as five stage process as which is evolved from the existing system [7]. In the first stage, the input image is preprocessed to get more accurate feature points. During the second stage, the chest walls of MR and MLO images are detected to remove the pectoral muscles. The above image is considered as third stage for nipple localization by finding out the minimal objectdistance from the coordinate system with the help of boundaries obtained from boundary tracing. The fourth stage is the detection of masses and micro-calcifications from MRI and Mammogram using image segmentation through region growing and thresholding. The last stage is the registration where the images are registered to get the most detailed image to obtain the accurate location of masses and micro- calcifications into single image. These stages are

   pictorially represented using block diagram shown in Fig. 1.

   Fig. 1. Block diagram of the proposed system

   1. Pre-processing Method

      The input images (Fig. 2) MRI and MLO mammograms are subjected to several processing methods before it is used for registration activity. The image pre-processing step [3] helps to increase the robustness of the system under varying conditions. Initially, the images are converted into gray scale images and resized into a specific size, 512×512. This helps to reduce computational complexity of the system.

      Fig. 2. The Input images (MRI and MLO mammogram)

      The MR images and MLO mammograms undergo filtering to remove the noise present in the image. Since the medical images contain black and white noise, the most

      efficient filtering used in MRI is median filter and in mammogram bilateral filtering is more suited to remove the noise. During noise removal some details in the image is lost; to reproduce that information without noise, edge enhancement method is applied. For this, Gaussian filtering and intensity adjustments are used. To obtain smooth surfaces, morphological operation [3] is performed. These techniques help to improve the detection accuracy of the breast from boundary.

   2. Chest wall Removal and Nipple Localization

      The pectoral muscle region [4] (in MLO mammograms) has a predominantly high density that may interfere with the analysis of mammograms. The prior detection and removal of this region could lead to improved results in lesion localization. The high density of the pectoral muscle results in strong edge separating breast area from chest wall. This strong edge can be approximated by a straight line. The region other than the breast area is removed to obtain the breast region alone. The pectoral muscle removal steps are as follows:

      1. Object Extraction

         Object extraction is used to extract the object (breast with chest wall) part from background for its use in pectoral muscle removal. In MR images, the object extraction is done through binary thresholding. Thresholding results in extraction of object area from the background. In MLO mammograms, gradient edge detection method is applied. These binary conversions will help in extracting the object area devoid of background.

      2. Boundary Tracing

         Boundary Tracing is performed to find out the boundary points of breast region in both images. From previous step, breast is extracted in the form of binary image and is ready for boundary tracing. Boundary tracing algorithm is used for drawing the boundary along edges of breast region. The boundary tracing begins from start point of the object boundary and traverse through edges of object by drawing curve. The obtained boundary points are used for drawing line segments.

      3. Pectoral Muscle Removal

         The pectoral muscles should be deleted from the breast image to avoid complexity in lesion identification. The node present in chest wall appears as lesion and they have similar intensity as that of cancerous lesion. The pectoral muscles in MR images and MLO are removed with the help of line segments drawn from boundary points. The boundary points obtained from boundary tracing is used for constructing line segments. The longest line segment that passes the boundary points is considered as the edge that separates pectoral muscles from breast area.

         The lift image toolbox provided by Matlab is used in MLO mammograms to remove the chest wall from the object. Lift image toolbox is useful in the case of irregular shapes with any number of edge points. Similarly, the breast area is obtained and

         is used for registration activity. The pectoral muscle removal in mammogram and MR images is illustrated in figures 2 and 3.

         Fig. 2. Pectoral muscle removal in mammogram: (a) plotting of line segments,

         (b) pectoral muscle detected and (c) chest wall removed.

         Fig. 3. Pectoral muscle removal in MR images: (a) plotting of line segments, (b) pectoral muscle detected and (c) chest wall removed.

      4. Nipple Localization

         In order to locate nipple position during registration, matching control point selection is the first step to be made. For this, some standard points be selected in which nipple location is the one among them. The boundary point with the least displacement from coordinating axis is generally considered as nipple. It is followed that the object is always aligned to the

         right of the image when locating nipple position. The lesion position is localized in terms of clock face position. The location of masses and micro-calcifications can be described by dividing the breast into four quadrants: the upper outer quadrant (UOQ), the upper inner quadrant (UIQ), the lower outer quadrant (LOQ) and the lower inner quadrant (LIQ). Each quadrant is further subdivided to locate the position of masses and micro-calcification using clock face model.

   3. Lesion Localization

      To identify the exact location of masses and micro- calcifications from MR images and MLO mammograms can easily be made when considered separately. To include both the information on a single image can be achieved only through registration method. The breast extracted image obtained from previous step provides highly detailed image carrying the information on masses present in MRI and micro-calcifications in MLO mammograms. The identified masses in MRI are segmented out using the region growing method and the micro-calcifications are identified using thresholding. Their location in breast can be calculated from clock face model.

      The positions of micro-calcifications in MLO mammogram are obtained from MLO and CC images together. The MLO view gives the information on upper or lower part of breast and the CC view provides the inner or outer region of breast where micro- calcifications are found. Including both the information the clock position of masses and micro-calcification can be obtained (Fig. 4).

      Fig. 4. The micro-calcifications are found to the right lower inner quadrant (LOQ) around 4 oclock position and to the left upper inner quadrant (UIQ) around 10 oclock position.

   4. Registration

      Image registration [5][6] is the process of overlaying two or more images of the same object taken at different times, from two modalities. It geometrically aligns two imagesthe base (MRI) and input (MLO mammogram) images. In proposed system, the medical images of breast taken from different imaging techniques are considered for registration activity. Selection of control points is the key factor in image registration. The control points used in the proposed model are the nipple position and the end points of the breast line. These points are automatically located from MRI slices and MLO images for its use in registration activity.

      The concept of registration is pictorially represented below (Fig. 5).

      Initially, matching control points are picked from MRI and MLO mammograms to align them in position. The control points in MRI slices are taken from multiple slices as the MRI slice containing nipple may not possess details of masses present. The visibility of masses in breast may depend upon the depth of occurrences. Also the MRI slice that is taken from inner breast region having masses may not contain the nipple position visible. In such cases, more than one slice is used for locating the control points. These points are marked in the image slice having masses are used for registration with MLO images. The registration overlaps the MRI and MLO images by applyin some spatial transformations. The different types of transformations used are projective, affine, etc. Some resampling methods are done to align the images during registration time and the resulting image gives better information on features in registered image. The registration algorithm for the proposed system is explained as:

      1. Consider input image I(x,y) and base image B(x,y)

      2. Use cpselect to select matching control points (CP)

      3. Pass CP to infer spatial transformation

      4. Apply spatial transform and register it with B(x,y)

   Fig. 5. Registration steps

4. RESULT AND DISCUSSION

   The data is collected from Regional Cancer Centre (RCC), Thiruvananthapuram, Kerala, India. The dataset composed of MRI and Mammogram images (MLO) of different patients. The MRI dataset consists of more than 728 slices from a single patient at different time interval: containing masses as well as lesion free slices. The number of slices containing lesion vary for different patients with different type of lesion present. The mammogram dataset consists of 6 images from a single patient. The image resolution of MRI is 512x512x3 and the size of the MR image is 116 kB. The mammogram (MLO) image resolution is 4096 ×3328 and the size varies from 226kB to 3.32 MB. The size and resolution can vary for different patients.

   The MR images and MLO mammograms undergo pre- processing steps such as: Gaussian filtering, edge enhancement and morphological operations, to obtain noise free edge enhanced smoothened image. After pre-processing, the chest wall removal and nipple localization is performed using gradient method and boundary tracing method. This pre- processed set is used for enhancing the quality of MRI and mammogram images. The registration method helps to localize the masses and micro-calcification into single breast image. The position of masses and micro-calcifications in breast is identified using clock face model.

   The proposed technique is implemented in the working platform MATLAB ® 2010 and it is evaluated using 10 cases having breast cancer as testing set. Among the 10 patient profiles, 7 profiles contain lesion, 2 are falsely missed lesion and 1 containing no lesion. The precision is obtained a value of 87.5%, i.e. it shows the correctness of the system in detecting the lesion from breast image. The recall is obtained as 77.8%,

   i.e. it denotes the probability that a lesion is detected and the F1 score is obtained as 82.375 % i.e. combines both precision n recall values.

   Fig. 6. Chest Wall Removal (a) MRI boundary with pectoral muscle detection,

   (b) chest wall removes MR image, (c) MLO boundary with pectoral muscle detection and (d) Chest wall removed MLO image.

   Fig. 7. Nipple localization

   calcifications from

   Fig. 8. Lesion segmented from MRI and micro-

   Mammogram

   Fig. 9. Registered Image

   Fig. 10. Lesion localization in registered image

   The results of lesion position detection using registration are shown in figures 6 to 10. All processing methods can have an error rate and in some occasion it may either fail to identify the lesion present or falsely identify the lesion that is not present. It is common to describe this error rate by the terms true and false positive and true and false negative.

   The registration works for images from clinical routine helps the experts in identifying the position of masses and micro-calcification. The accuracy in locating their position in breast shows promising results even when the clinical variability of datasets is large. The obtained results are in the same range as the accuracies achieved in literature though in this work fully automated approach was used. This method is able to register images from different modalities with MLO mammogram. The registration does not take gravity into account. This is motivated from other literatures reporting insignificant influence of gravity in diagnoses.

   Lesion positions were not used as landmarks for registration. The patient profile set included a large variety of lesions, either in size, location or contrast enhancement. Image similarity based optimization seems not to be influenced by the nature of the lesion. This leads to the conclusion that registration approach in the proposed system does not strongly depend on the type of lesion and the visibility of lesion. The nipple position and the points near the breast and the chest walls are used as standard points for registration.

5. CONCLUSION

In this paper, an automatic method for positioning masses, micro-calcifications in MRI and MLO mammograms using registration method is proposed. This approach helps in localizing the position of masses and micro-calcifications in breast from a large number of clinical datasets. The MLO view of Mammogram and the MR images are used for locating the masses and micro-calcification present with the help of registration activity. In the proposed system, the test images undergone Gaussian filtering, edge enhancement and

morphological operation resulted in enhanced data images for computational operations. The chest wall removal and nipple localization is successfully achieved through boundary tracing method. The different test data are correctly overlapped in one coordinate system through registration activity. The registered images provide information as a combination of details obtained from MRI and MLO mammograms. The position of masses in MRI and micro-calcification in MLO mammograms are included in a registered image helps medical practitioners to examine the details to provide better treatment to the patients.

REFERENCES

1. Nicole V. Ruiter, Rainer Stotzka, Tim-Oliver Müller, Hartmut Gemmeke, Jürgen R. Reichenbach, and Werner A. Kaiser, Model- Based Registration of X-Ray Mammograms and MR Images of the Female Breast,IEEE Transaction on Nuclear Science, Vol. 53, No. 1, 2006.

2. C. P. Behrenbruch, M. Yam, M. Brady, and R. E. English, The use of magnetic resonance imaging to model breast compression in X-ray mammography for MR/X-ray data fusion, in 5th Proc. Int. Works. Digital Mammography, Toronto, Canada, pp. 688696, 2000.

3. T. Hoppa, M. Dietzelb, P. A. Baltzerc, P. Kreiseld,W. A. Kaiserd, H.Gemmekea, N. V. Ruiter , Automatic Multimodal 2D/3D Breast Image Registration using Biomechanical FEM Models and Intensity- Based Optimization, Medical Image Analysis, vol.17, pp. 209-218, 2013

4. S. K. Kinoshita, P. M. Azevedo-Marques, R. R. Pereira Jr, J. A. H. Rodrigues, and R. M.Rangayyan, Radon-Domain Detection of the Nipple and the Pectoral Muscle in Mammograms, Journal of Digital Imaging, Vol 21, No. 1 (March), 2008.

5. Barbara Zitova, Jan Flusser, Image registration methods: a survey, Image and Vision Computing, Vol. 21, issue 11, pp. 977-1000, 2003.

6. Tran Duy Linh, Huynh Quang Linh, Department of Biomedical Engineering Faculty of Applied Science HCMC University of Technology, Vietnam, Medical Image Registration In Matlab.

7. Torsten Hopp, Pascal Baltzer, Matthias Dietzel, Werner A. Kaiser, Nicole V. Ruiter, 2D/3D Image Fusion of X-ray Mammograms with Breast MRI: Visualizing Dynamic Contrast Enhancement in Mammograms, International journal of Computer Assisted Radiology and Surgery, Vol. 7, issue 3, pp. 339-348, 2012.

8. [Online]. Available: www.radiologyinfo.org