Dr Gabor Fichtinger is Chair of Cancer Care Ontario Research and Professor at Queen’s University School of Computing in Kingston, Ontario. His interests are in computer-assisted surgery and interventions. Dr Jan Fritz is the Director of Interventional MR Imaging and an Assistant Professor of Radiology and Radiological Science at Johns Hopkins University School of Medicine, Baltimore, Maryland. The two have been collaborating to create a computerised system that overlays 2D magnetic resonance (MR) images onto a patient during surgical procedures, to improve needle placement.
Percutaneous procedures, which take place beneath the skin, are common in medicine. Using needles for such procedures requires considerable clinical acumen, and can take several attempts, leading to discomfort for the patient and extending the duration of the procedure.
Image-guided systems making use of computed tomography (CT) or magnetic resonance (MR) imaging can help doctors by providing tracking of the location of the needle, allowing them to ‘see’ beneath the skin more accurately (with what Fichtinger and co-authors call ‘X-ray vision’). However, new images must be taken for every insertion of the needle, so if it is not done perfectly the first time, this can greatly extend the length of a procedure. The aim is to be able to insert a needle in one go, thereby eliminating potential complications from multiple reinsertions. This would minimise the amount of human inaccuracy involved, and reduce the time and effort required to take new CT/MR images to guide insertion.
Overlay images can help guide the placement of the needle. Overlay imagery involves acquiring an image slice, adjusting the magnification and alignment of the image, and then displaying it where the physician can see it. This allows in situ visualisation. Overlay imagery can be used for a wide range of treatments such as biopsies or musculoskeletal interventions for the shoulder, hip, and spine, as well as in procedures such as the removal of kidney stones (nephrolithotomy).
Static Image Overlay Systems (SIOS) were developed to improve upon manual or CT/MR-based methods of needle placement. These systems use mirrors and monitors that are configured in a ‘viewbox’, so that the clinician can see a virtual image that appears to float at the correct position inside the patient.
The problem with SIOS
A weighty, counterbalanced mechanical arm, which holds the viewbox, is required for previous SIOS configurations, which are fixed either to the floor or the CT/MR scanner. This limits the range of motion possible and gets in the way of the practitioner. Additionally, the equipment takes up a lot of room, occupying valuable space in treatment rooms, and limiting precise movement. It can easily be knocked and requires careful calibration, which can limit its usefulness and ease of use.
MIOS to the rescue
Mobile Image Overlay Systems (MIOS) can be used with a tablet, so have a greatly reduced system weight, smaller dimensions, and improved tracking accuracy. The weight of an MIOS is eight times less than an SIOS, meaning the heavy counterbalanced arm is not needed, and its tracking is five times more precise. A table-mounted version is also available, which professionals found easier to use.
How it works
Previous incarnations of the MIOS were less easily calibrated and less accurate, but Drs Fichtinger and Fritz have subsequently improved the angle of the display, developed a self-calibrating model, and validated the tool to show that the error in precision is minimal.
The viewbox is an integral part of MIOS. This is a handheld device that attaches the tablet to a mirror and the optical markers that are used to guide the image and calibrate the system. Optical markers on the side of the viewbox position the needle in the virtual image accurately, and markers on the patient track its position in the real world. These markers allow the position of the needle to be tracked precisely in three dimensions. The system uses open-source software to create ‘slices’ from these marker positions that are visualised on the tablet.
Optimisation and validation
Image overlay systems have been tested throughout their development. Drs Fichtinger and Fritz have tested various versions of the technology using phantoms (inanimate subjects, such as a melon or a full-size replica of the human body), cadavers, and real-life human studies. They have also performed usability tests, where practitioners have made use of the technology and evaluated its application to their work. From this latter study, they concluded that doctors found the technology useful, though the table-mounted version was easier to use than the hand-held tablet device.
The team has found that the optimum angle between the mirror and monitor is 90°, rather than the 60° that had previously been used. They have also improved the type of laser used and its alignment with the virtual image overlay.
Further, they have developed a calibration method. Two-step calibration takes place away from the patient, and is only required once. The first step is the system calibration, which requires three optical markers. The system is moved through space, and the changes, or transforms, that are registered are recorded using these trackers. These transforms can then be compared and the system can be calibrated accordingly.
The second step is to calibrate the system to each individual patient by overlapping known points on the patient’s body with the same points in the scanned image. The software also self-calibrates using trackers attached to the patient and viewbox, which reports the position of the patient in relation to a visualised virtual image plane.
Overall, the team has validated MIOS and shown that it is accurate enough to be a clinically useful tool.
Use with CT and MR
More experiments have shown that the use of overlay imagery with CT or MR can provide accurate needle placement by reducing the number of steps and rounds of imaging required, and offering a visual guide that improves the chances of achieving accurate needle placement in the first attempt.
Use in training
Further, a validation system called ‘Perk Station’ has been developed that is designed for training and evaluation. Studies show that medical students who were trained using the Perk Station software performed needle insertions more accurately than those who performed freehand insertions.
The team has made the code and blueprints freely available and open source, so that others can easily replicate and adapt the software and hardware. This is especially useful for training medical students and professionals who need to perfect their technique for these incredibly difficult and highly skilled procedures.
- Research Objectives
Dr Fritz and Dr Fichtinger’s research predominantly focuses on computer-assisted surgery and interventions. This includes areas such as medical imaging, image computing, scientific visualisation, surgical planning and navigation, and the integration of these within clinical environments for clinical use.
National Institutes of Health (NIH) R01 CA118371; The Natural Sciences and Engineering Research Council of Canada; Cancer Care Ontario
Dr Fichtinger’s academic life began in Hungary where he studied at the Technical University of Budapest. He is now the Professor and Cancer Care Ontario Research Chair in the School of Computing at Queen’s University, and an Adjunct Professor of Computer Science and Radiology at Johns Hopkins University. Jan Fritz, MD is an Assistant Professor of Radiology and Radiological Science at Johns Hopkins University and specialises in Musculoskeletal Radiology, having previously studied at the University of Tübingen in Germany.
Gabor Fichtinger, PhD, Professor
Cancer Care Ontario Research Chair
School of Computing
557 Goodwin Hall, 25 Union St.
Kingston, ON, K7L 2N8, CANADA
T: +1 613 533 6050
Jan Fritz, M.D., P.D., D.A.B.R.
Director of Interventional MR Imaging
Assistant Professor of Radiology and Radiological Sciences, Johns Hopkins University School of Medicine
Russell H. Morgan Department of Radiology and Radiological Science
601 N. Caroline Street, JHOC 3140A
Baltimore, MD 21287, USA
- Using technology to improve needle procedures