Image Fusion of Ultrasound with Different Imaging Modalities using Virtual Navigator

Image Fusion of Ultrasound with Different Imaging Modalities using Virtual Navigator

Aruna.N.L (MTech), Meharunissa.S.P (Researcher), Parameshwar Reddy (MTech)

Department of Instrumentation Technology, Dayananda Sagar College of Engineering (Bangalore),

Vishvesharaya Technological University, Belgaum, Karnataka, India nlaruna90@gmail.com meharbhakshi2013@gmail.com parameshec72@gmail.com

Abstract Image fusion is the combination of two different imaging modalities with regard to one and the same anatomic area. The intent is to combine anatomic and functional image information. The Virtual Navigator is an advanced system allows the real time visualization of transcranial ultrasound images with other reference images ex: CT (computed tomography), MRI (Magnetic Resonance Imaging), and PET (Positron Emission Tomography). Here we are using Brain MRI, Brain CT and Brain PET scanning techniques as a reference image and we are comparing all the three imaging modalities fusion. The combination has, as the final result, the real time data fusion which allows increasing the accuracy and confidence of ultrasound scanning, by overlaying the different images or visualizing them side by side.

The superimposition of US to the previously acquired MRI, CT and PET volume consisted of two procedures depending on the operator skill and experience. One is external marker registration and another is internal marker registration. In external marker registration we are using a point based rigid registration. The internal marker registration is obtained by scanning from any available transcranial ultrasound window. The common registration used is external fiducial marking acquired with the two modalities was improved using facial anatomical landmarks. By doing all these things we get which fused image will give better performance, high resolution picture, and increases clinical diagnostic.

Keywords: Image Fusion, Virtual Navigator, Magnetic Resonance Imaging, Computed Tomography, Ultrasound, Positron Emission Tomography.

1. INTRODUCTION

   In 2002 Esoate designed and introduced new system dedicated to fusion imaging modality, the Virtual Navigator (VN) [1], [4], specially focused on interventional imaging. The need of developing such an advanced system is because of the intrinsic characteristics of the Ultrasound imaging modality. Ultrasound is the most widely diffused imaging modality for real time guidance / monitoring of diagnostic and therapeutic interventional procedures in all their phases, also due to its nearly universal availability, portability, ease of use and cost effective features. However, Ultrasound has also have some limitations like insufficient sensitivity for the detection of small lesions in some conditions, field of over view and low resolution of image. In recent years real time Ultrasound image fusion with a pre-acquired second imaging dataset Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) [10] has become widely used both for diagnostic purposes and image guided interventional procedures especially in the liver and in prostate as well as for a better localization of tumour and bone landmarks in osteoarthritis. VN advanced image fusion system is the best assistant to physicians during any phase of the interventional procedures.

   Ultrasound has been used to image the human body for over half a century. Dr. Karl Theo Dussik, an Austrian neurologist, was the first to apply ultrasound as a medical

   diagnostic tool to image the brain. Ultrasound (US) is one of the most widely used imaging technologies in medicine. It is portable, free of radiation risk, and relatively inexpensive when compared with other imaging modalities, such as magnetic resonance and computed tomography. Modern medical US is performed primarily using a pulse-echo approach with brightness – mode (B-mode) display. US waves have frequencies that exceed the upper limit for audible human hearing, i.e., greater than 20 kHz. Medical ultrasound devices use sound waves in the range of 120 MHzs. High- frequency ultrasound waves generate images of high axial resolution. Low-frequency waves offer images of lower resolution but can penetrate to deeper structures due to a lower degree of attenuation. For this reason, it is best to use high- frequency transducers (up to 1015 MHz range) to image superficial structures (such as for stellate ganglion blocks) and low-frequency transducers (typically 25 MHz) for imaging the lumbar neuraxial structures that are deep in most adults. US are relatively inexpensive, portable, safe, and real time in nature.

   Magnetic Resonance Imaging (MRI) [10], [5] is a medical imaging technique used in radiology to detect the anatomy and function of the body in both health and disease. MRI uses only magnetic fields. The field strength is measured in tesla and majority system operates at 1.5T, commercial systems are available between 0.2T to 7T. An MRI scanner can create clear, detailed pictures of the structure of the brain and can detect any abnormalities or tumours. Sometimes a dye, or tracer, such as gadolinium may be introduced via a vein in the arm, to improve contrast in the image. The process of having a scan is painless and safe, there is no exposure to radiation but occasionally, a patient may have a reaction to the tracer dye. Pregnant mothers are not recommended to undertake the procedure unless there is no alternative, since it is not known whether the effects of a strong magnetic field may affect the developing baby. Higher the Tesla it will give better quality picture. The only limitation of this MRI is the anatomical image lacks of functional information and it takes more time to scan a part i.e. from 30 to 45min.

   Computed Tomography (CT) [10], [3] is a Non-Invasive diagnostic imaging procedure that uses a combination of X- rays and computer technology to produce horizontal or axial images of the body. The development of the first clinical CT began in 1967 with Godfrey N. Hounsfield at the central research laboratories of EMI, Ltd, in England. While

   investigating pattern recognition techniques, he deduced, independently of Cormack, that X-ray measurements taken through a body from different directions would allow the reconstruction of its internal structure. The first laboratory scanner was built in 1967. The first clinically available CT device was installed at Atkinson-Morley Hospital in September 1971, after further refinement of the data acquisition and reconstruction techniques. Images could be produced in 4 and half minute. On October 4, 1971, the first patient with a large cyst was scanned and the pathology was clearly visible in the image. For their pioneer work in Computed Tomography Cormack and Hounsfield shared the Nobel Prize for Physiology and Medicine in 1979.

   Positron Emission Tomography (PET) [10] was introduced clinically in 1970s, PET provided a fundamentally new opportunity to explore the parts of the brain that were activated in undertaking specific tasks, a role it dominated for more than a decade. PET was the first major technology to measure physiological functioning in the brain. In PET scanning, the regional distribution of exogenously administered positron emitting tracers is measured using tomographic imaging. The first PET tracer to be used in humans was 18F deoxyglucose, which distributes according to regional glucose utilization. Because water is freely diffusible from the blood to the brain, 15O-H2 O provides a PET tracer for measuring cerebral blood flow and was another early tracer used for measuring regional brain function. More recently the main functions for PET are focuse on the study of neurotransmitter, the actions of pharmaceutical drugs and the expression of specific genes in the brain.

2. METHODOLOGY

   1. Obtaining input images

      Virtual Navigator (VN) advanced technique allows the real time visualization of ultrasound images with other reference imaging modalities (CT, MRI, and PET). Here we are using Transcranial Ultrasound as source image and Brain MRI as reference image, by combining these images we will get better accuracy, increases the reliability of the diagnostic process and gives the high resolution of the image. The VN unit had an electromagnetic tracking system, consisting in a transmitter on a fixed position and a small receiver mounted on the US probe through a reusable tracking bracket.

      BLOCK DIAGRAM

      Fig.1: Block diagram of comparing the efficiency of all three image combination using MATLAB.

      US uses PA240 phased array probe, it offers extraordinary comfort during the examination for both patients and operators. Even difficult to scan patients wont feel any discomfort usually caused by the intense probe pressure on the ribs. This PA240 probe is mounted through a reusable tracking bracket 639-039, CIVCO medical solutions. The transmitter, whose position was considered the origin of the reference system, was kept steady and correctly oriented towards the patients head by a proper support, while the receiver provided the position and orientation of the US probe in the 3D space, in the system of reference given by the transmitter. The electromagnetic tracking system generates a magnetic field which has the highest magnitude in the position of the transmitter, so the latter was positioned at a distance of 10cm from the subject head.

      Fig.2: a) PA240 US phased array probe. b) 639-039 CIVCO medical solutions tracking bracket.

      MRI acquisitions were performed using 1.5T scanner with a maximum gradient strength of 33 mT/m and slew rate of 125 mT/m/ms, using standard 12 channels matrix head coils. This MRI scanner uses a strong magnetic field and radio waves to create pictures of the tissues and other structures inside the brain, on a computer. The magnetic field aligns the protons in hydrogen atoms, like tiny magnets. Short burst of radio waves are then sent to knock the protons out of position, and as they realign, they emit radio signals which are detected by a receiving device in the scanner. The signals emitted from the different tissues vary and they can be differentiated or distinguished in the computer picture. An MRI picture can create clear detailed picture of the brain and can detect any abnormalities or tumours. Sometimes a dye or tracer such as gadolinium may be introduced via a vein in the arm, to improve contrast in the image. The relaxation times T_1, T_2 and T_2* are measured after the scanners pulse sequence, and can be chosen to look at specific tissue within the brain. It may be seen that by selecting different relaxation times and manipulating radio frequencies, specific brain tissue can be highlighted for examination by the physician.

      The following sequences were acquired: I) Scout T1 sequence: three sagittal slices, three coronal slices and one axial slice with low resolution, used for positioning and orientation of the other sequences. II) Axial Proton Density turbo spin echo, with the following parameters: TR, TE, echo train length, flip angle, slice thickness, matrix size and field of view. A slab of 100 slices without gap was acquired; the central slice of the slab was positioned parallel to the line that joins the inferior-anterior and inferior-posterior edge of the corpus callosum, visible on the sagittal scout T1. This standard

      positioning guaranteed the inclusion of the whole brain and the skin with the 6 fiducial markers.

      The process of having a scan is painless and safe, there is no exposure to radiation, but occasionally, a patient may have a reaction to the tracer dye. Pregnant mothers are not recommended to undertake the procedure unless there is no alternative, since it is not known whether the effects of a strong magnetic field may affect the developing baby. The scanner is a large tunnel surrounded by a circular magnet. The patient lies on a couch which slides into the tunnel. It is quite noisy so the patient is given headphones with music of their choice, and has to keep still for 30mins to 45mins as the tiny radio wave signals are picked up by the computer. It is entirely painless, but children may require a general anaesthetic to keep them still for long enough. The radiographer will need to know if the patient has any metal in their body such as metal skull plate, inner ear implants, pacemaker, artificial joints, screws and pins holding bone fracture repairs. The patient may resume normal activities immediately after the scan and the radiologist studies the pictures and sends a report to doctor.

      Computed Tomography (CT) has been one of the biggest breakthroughs in diagnostic radiology. The first clinical CT scanner was developed by Godfrey N. Hounsfield for examinations of the head and was installed in 1971 at Atkinson-Morleys Hospital in Wimbledon, England. The first body CT scanner was installed in 1974, and before the end of the 1970s the basic technical evolution of CT was complete. The latest innovation is the introduction of multi-slice CT in 1998. This new technology is vastly expanding the performance of CT scanners. It truly transforms CT from a transaxial imaging modality to a 3D technique that yields high quality images in arbitrary planes and forms the basis for an expanding variety of 3D visualization techniques, including virtual endoscopy.

      Computed Tomography (CT) is an X-ray Tomographic technique in which an X-ray beam passes through a thin axial section of the patient from various directions shown in Fig.1. Parallel collimation is used to shape the X-ray beam to a thin fan, which defines the thickness of the scan plane. Detectors measure the intensity of the attenuated radiation as it emerges from the body. A mathematical image reconstruction (inverse radon transformation) calculates the local attenuation at each point within the CT section. These local attenuation coefficients are translated into CT numbers and are finally converted into shades of gray that are displayed as an image. With conventional CT scanners the volume of interest is scanned in a sequential fashion, usually proceeding one section at a time. CT scanning of the head is typically used to detect infarction, tumours, calcifications, Haemorrhage and bone trauma. Hypodense (dark) structures can indicate edema and infarction, hyperdense (bright) structures indicate calcifications and hemmorrahge and bone trauma can be seen as disjunction in bone windows. Tumours can be detected by the swelling and anatomical distortion they cause, or by surrounding edema.

      Physiological imaging techniques measure changes in cerebral blood flow (CBF) and brain metabolism. These measurements complement structural imaging studies by providing information on regional brain function either at rest or in response to specific perturbations and how it is altered by brain disorders. Positron emission tomography (PET) is a nuclear medicine, functional imaging technique that produces a three-dimensional image of functional processes in the body. The system detects pairs of gamma rays emitted indirectly by a positron -emitting radionuclide (tracer), which is introduced into the body on a biologically active molecule. Three- dimensional images of tracer concentration within the body are then constructed by computer analysis. The areas of the brain that command the greater volumes of blood produce the most gamma-rays and these areas that are computed and displayed by the PET scan. As the tracer decays, there is a point when gamma photons are emitted almost opposite to each other, the timing of this event is detected and will ultimately improve the detail of the image. This system not only identifies the activated area of the brain, but also measures the degree of activity./p>

   2. Image Registration

      Image registration is the process of determining the optimal spatial transformation that brings two images into alignment with each other. Image registration is necessary, for example, when images of the same object are taken at different times, from different imaging devices, or from different perspectives. Applications of image registration include image-guided radiation therapy (IGRT), image-guided surgery, functional MRI analysis, and tumour detection, as well as many non-medical applications, such as computer vision, pattern recognition, and remotely sensed data processing. [6], [7].

      For each subject of the study, the registration procedure of US space to MRI/CT/PET [2], [8] space included two steps: 1) a semi-automatic point based rigid registration of corresponding fiducial markers; 2) a manual (image based) fine tuning, based on anatomical landmarks localized on the basis of the previous coarse registration.

      Step.1: Point based rigid registration is commonly used for image guided systems. One set of point is to be registered to another set of corresponding points by means of a rigid registration of the first set. For neurosurgery, because of the rigidity of the skull, the point mapping is typically a rigid registration. The registration is usually based on the fiducial markers that are attached to the head before imaging and remain attached until the procedure begins. A fiducial point set is obtained by localizing each fiducial marker both in the image and in the operating room. As regards the first one, the fiducial markers on the MRI/CT/PET rendering and those on the subject laying in the bed for US examination had to be manually identified, for recording their coordinates in the systems of reference and for establishing the correct correspondence between them. Finally, the registration matrix between the two stereotaxic spaces was automatically computed by the VN tool, using the markers coordinates in the two systems. More in practical detail, the first phase consisted on locating the external fiducial markers on the subjects

      forehead (fig: 3, right): the coordinates of every fiducial marker in the US system of reference (P_pen) were obtained with a registration pen equipped with the electromagnetic receiver (fig: 3, middle). In the same order, each in the same order, each marker was also located in the MR image reference system: the markers were manually identified on the 3D Proton Density rendering to collect their coordinates (P_MR).

      Step.2: The registration of the US image to the MRI/CT/PET was then improved with a manual fine tuning, aiming at reducing the misalignment between the MRI/CT/PET and the registered (with M_reg) US image, particularly in the internal structures, near the vessels which will be investigated. Indeed, a minimum shift between the external fiducial markers on the two imaging modalities could cause a higher internal shift that we want to minimize. This was obtained by selecting structures which are well enhanced in both the image modalities, due to either anatomical or hemodynamic contrast, or manually reducing their mutual shift. The internal structures considered for the image-based registration were: the third ventricle and the mid brain visualized with B-mode US from the temporal window, the Circle of Willis and the Middle Cerebral and Posterior Cerebral arteries, visualized with CD from the temporal transcranial window (Fig. 4); bone structures (the petrous apex and the sphenoid bone), insonated through the condylar window. The procedure required the user to identify the anatomical landmark on the 2D B-mode or CD US and to manually match it with the corresponding structure in MRI, freezing and manually shifting the US plane on the MR image until no residual shift was observed. This procedure was repeated for different scan orientations in order to guarantee a good match in the 3D space.

   3. Transcranial ECD Examination And Navigation Procedures

      Fusion imaging procedure with VN was performed in supine posture (0 tilt). Non-metallic beds were used to avoid interference with the electromagnetic VN system. TCCD examinations were performed by an expert sonographer (more than 20 years of experience in US examination) trained for the US venous transcranial evaluations through temporal and condylar windows (3 years of experience for this new approach). Each subject underwent trans-condylar TCCD evaluation at the level of the condyloid process of the mandible (Fig. 4(b)). In order to assess errors introduced by the conventional blind TCCD examination without VN embedded on US system, the DCV targeting and CD evaluation procedures were performed with the MRI frame switched off, thus based on the sole B-mode. If no signal was detectable due to low flow velocity, the subject was asked to take a prolonged and deep inspiration; with the maximum duration he/she was capable to take, in order to increase the venous flow velocity and to be able to locate the addressed vein with CD signal. If neither this was successful for finding the vessel of interest, it was considered not seen. Finally, the MRI volume was switched on and the anatomical position of the targeted vessel was assessed and confirmed with the color Doppler fusion with MRI. [9]

   4. FIGURES

   Fig.3: Registration: step 1. First phase: pointing of the corresponding fiducial marker both in the Proton Density MRI/CT/PET 3D rendering (b), with the 3 main planes as reference (coronal -a-, axial -c-, sagittal -d-) and on the subjects forehead (g) laying in the bed before US examination. (f) Registration pen.

   Fig.4: (a) Rendering of proton density MRI/CT/PET, with arrows pointing the temporal and condylar windows. (b) Probe position for the US

   examination through temporal and (c) condylar windows.

3. RESULTS

   The system repeatability error measured with the registration pen was always under 0.1 cm.

   Registration Quality with Fiducial Markers

   For all the performed examinations, the registration error obtained after the first step of the co-registration process was below 0.5 cm. An example of the residual external and internal shift after the step 1 of the registration with fiducial markers is shown in Fig. 5.

   Fig.5: (a) Example of US image, insonation from the condylar window;

   (b) MR oblique section corresponding to the US image after the first registration step; (c) fusion of (a) and (b).

   Examples of the final registration performance (using fiducial marker and image-based refinement) are shown in Figs. 6(a), 7, and 8, where the fusion of the midbrain (Fig. 6(a)) and of the middle (Fig. 7) and posterior cerebral arteries

   (Fig. 8) are representing the good internal matching between the two imaging modalities.

   Fig.6. Midbrain (a) and sphenoid bone (b) visualized with the two imaging modalities. Their fusion gives information about a good internal registration quality. In both examples there is a perfect matching, i.e. zero

   visible shift of the internal structure of interest visualized with the two imaging modalities.

4. CONCLUSION

   The present study on TCCD fusion with MRI/CT/PET, based on a VN technology embedded on a US system and an improved registration procedure, has allowed comparing the efficiency and accuracy of the results of these three different image fusion modalities, even those that are not detectable with the common used US windows. The proposed two step registration method firstly based on external fiducial markers or facial landmarks, then on internal structures, demonstrated good applicability and precision. Indeed, the first step of the registration procedure was acceptable for all the subjects; the second step always reduced the registration error of internal structures to the minimum visible with human eyes. By combining or fusing the US-MRI, US-CT, US-PET using 1st and 2nd registration step we can reduce the error and misalignment between the images, then we can compare their accuracy.

5. ACKNOLEDGMENT

   The authors would like to thank the Gururaj.Rao, Radiologist, and Karnataka, India. And also for thank for Vishvesharaya Technological University for supporting this research.

6. REFERENCES

Fig.7 Internal vessel landmarks (middle cerebral artery) used for the manual registration adjustment and for checking the final registration quality.

The US image is shown in (a), the MRI is shown in (b) and the fusion is shown in (c) and (d) with progressive decrement of US transparence.

Fig.8: a) Brain CT image b) US image of circle of Willis

c) image fusion of US and CT using image registration method.

Fig.9: a) US image of circle of Willis, (b) Parkinsons disease image using PET scan, (c) image fusion of (a) and (b) images using image registration.

1. S. De Beni, M. Macciò, and F. Bertora, Multimodality navigation tool

   Navigator, in Proc. IEEE SICE 2nd Int. Symp. Measurement, Analysis and Modeling of Human Functions, Genova, Italy, Jun. 2004.

2. L. Forzoni, S. DOnofrio, S. De Beni, M. M. Laganà, V. Kolev, G. Baselli,

   G. Ciuti, and D. Righi, Virtual navigator tridimensional panoramic imaging in transcranial application, Biomed Tech, vol. 57, no. Suppl.1, 2012, 10.1515/bmt-2012-4282.

3. Esoate creativity in health care Virtual Navigator for Real-Time PET- CT/Ultrasound Fusion Imaging.

4. Esoate creativity in health care Virtual navigator neurosonology.

5. Y. Ge, V. M. Zohrabian, and R. I. Grossman, Seven-Tesla magnetic resonance imaging: New vision of microvascular abnormalities in multiple sclerosis, Arch. Neurol., vol. 65, pp. 812816, Jun. 2008.

6. Dana Paquin, and Doron Levy, Department of Mathematics, Stanford University Stanford, CA 94305-2125 , Lei Xing, Department of Radiation Oncology, Stanford University Stanford, CA 94305-5947 Hybrid Multiscale Landmark And Deformable Image Registration.

7. Three-Dimensional Registration and Fusion of Ultrasound and MRI Using Major Vessels as Fiducial Markers Brian C. Porter, Deborah J. Rubens, John G. Strang, Jason Smith, Saara Totterman, and Kevin J. Parker*, IEEE transactions on medical imaging, vol. 20, no. 4, April 2001.

8. Leonardo Forzoni, Sara DOnofrio, Stefano De Beni, Maria Marcella Laganà, David Skoloudik, Giuseppe Baselli and Pietro Cecconi, Esaote S.p.A.,Fondazione Don Carlo Gnocchi ONLUS, IRCCS S. Maria Nascente, Department of Neurology, University Hospital Ostrava, Bioengineering Department, Politecnico di Milano, Firenze/Italy, Milano/Italy, Ostrava/Czech Republic. virtual navigator registration procedure for transcranial application, Proceedings of the IASTED International Conference Biomedical Engineering (BioMed 2012), February 15 – 17, 2012 Innsbruck, Austria.

[9]. M.M Laganà, L Forzoni, S.DOnofrio, S.De Beni, A Barberio, G Baselli and P Cecconi, Fondazione Don Carlo Gnocchi ONLUS, IRCCS S. Maria Nascente. Milano, Italy; Esaote S.p.A., Genova, Italy; Dipartimento di Bioingegneria, Politecnico di Milano, Milano, Italy. Assessment of the Cerebral Venous System velocities using Virtual Navigator Technology and Magnetic Resonance Imaging Proceedings of the 7th International Workshop on Biosignal Interpretation (BSI2012).

[10] Brain Imaging Technologies and Their Applications in Neuroscience, By Carolyn Asbury, Ph.D. With appreciation to John A. Detre, M.D., Ulrich von Andrian, M.D., Ph.D., and Michael L. Dustin, Ph.D. for their expert guidance.