3D optical imagery for motion compensation in a limb ultrasound system

Bryan J. Ranger, Micha Feigin, Xiang Zhang, Al Mireault, Ramesh Raskar, Hugh Herr, Brian AnthonyProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2016 · DOI

TL;DR

This paper introduces a novel multi-modal system for imaging residual limbs using ultrasound and 3D optical data. The system aims to improve prosthetic socket design by providing more accurate and quantifiable limb geometry, overcoming the subjectivity and discomfort associated with current methods. Preliminary results demonstrate the feasibility of this approach for creating compound ultrasound images with motion compensation.

What Problem Does This Paper Address?

The fabrication of prosthetic sockets is currently a highly subjective and manual process, leading to discomfort for a vast majority of amputees. Existing methods often lack quantitative patient-specific data, resulting in interfaces that are not optimally functional or comfortable. While advanced imaging techniques like MRI and CT have been explored, they can be expensive and have limitations. This paper addresses the need for a more accessible, cost-effective, and accurate method to capture the geometry and tissue properties of residual limbs for improved prosthetic socket design. The challenge lies in acquiring high-resolution volumetric data of a limb, especially considering natural limb motion during scanning.

Methods

The core of the system is a multi-modal approach combining ultrasound imaging with 3D optical tracking for motion compensation.

  • Ultrasound Imaging: A standard medical ultrasound probe is mounted on a rotating ring bearing within a water bath. This setup allows for circumferential scanning of the limb at set angular increments. The water bath serves to couple the ultrasound waves and maintain a consistent environment.
  • 3D Optical Motion Compensation: A Kinect™ One camera is positioned below the tank, capturing depth (point cloud), IR, and RGB images. This camera tracks the limb and probe's position in 3D space.
  • Motion Compensation Algorithm: The captured 3D point clouds of the limb are processed to filter noise. The Iterative Closest Point (ICP) algorithm is then applied to these point clouds to determine the transformation (rotation and translation) due to limb motion between consecutive ultrasound image acquisitions. This transformation is crucial for registering and stitching the ultrasound images into a coherent compound image slice.
  • Image Registration: The ultrasound images are initially placed at their acquired angular position. The ICP-derived transformation is then used to adjust their positions. For overlapping regions, the maximum value pixel is selected to create the final compound image slice. The probe's location and orientation in the IR image are also determined using a sum of square difference (SSD) comparison with a template.

Datasets

The paper presents preliminary results from scans of one amputee subject's residual limb and one non-amputee subject's arm. For comparison and ground truth, UTE-MRI images of the same subjects' limbs were acquired. No new benchmarks were introduced; rather, the system's output is compared to existing MRI data.

Key Insights and Ideas

  • Multi-modal Fusion for Robustness: Combining ultrasound for internal tissue detail with 3D optical tracking for motion compensation offers a powerful synergy. Optical tracking effectively addresses a key limitation of previous ultrasound-only systems, where limb motion degraded image quality.
  • Cost-Effectiveness Potential: By using a conventional ultrasound probe and a readily available 3D camera, the system has the potential to be significantly more cost-effective than MRI-based approaches for limb imaging.
  • Addressing Subjectivity in Prosthetics: The move towards quantifiable, patient-specific data from ultrasound imaging is a critical step in moving away from subjective prosthetic socket design, promising improved comfort and function for amputees.
  • Novel Mechanical Design: The integrated system with a rotating ring bearing and adjustable probe mount allows for automated data acquisition and flexibility in scanning different limb sizes and orientations.

Key Results

  • The system successfully captured 3D optical data (color, IR, depth) of the limb and probe.
  • Preliminary results show that the motion compensation framework, using 3D optical data, effectively reduces blurring artifacts caused by patient motion in the ultrasound images.
  • Compound ultrasound image slices of a human arm and residual limb were generated, showing anatomical structures and overall limb geometry comparable to MRI scans.
  • The system demonstrated the feasibility of creating a single compound axial image slice of a residual limb.

Limitations and Future Work

  • Limited Data Set: The study presents preliminary results from only one arm and one residual limb scan. Future work will involve reviewing a larger cohort of patients and comparing with their respective MRI data for further validation.
  • MRI-US Slice Correlation: The MRI slices used for comparison did not necessarily correlate precisely with the axial slices created by ultrasound. Future studies plan to use MR surface fiducial markers to ensure direct comparison.
  • Optimization of Ultrasound Parameters: The ultrasound probe was used with default parameter settings, meaning penetration depth, focus, and resolution were not optimized for the water tank setup. Future scans will focus on optimizing these parameters to achieve better depth resolution.
  • Full Volumetric Imaging: The current work focuses on single axial slices. Future iterations aim for automated acquisition schemes to enable volumetric scanning of the entire limb, reducing scan time.
  • Advanced Registration: More advanced registration algorithms for 3D motion compensation and edge detection for creating 3D skin and bone surfaces will be developed.
  • Decoupling Probe: The ultimate goal is to decouple the ultrasound probe from the mechanical positioning system, allowing for free-hand ultrasound image collection and stitching into a volume. This would provide clinicians with a powerful tool for quantitative data collection for prosthetic socket design and other clinical applications.

Summary updated

Abstract

Conventional processes for prosthetic socket fabrication are heavily subjective, often resulting in an interface to the human body that is neither comfortable nor completely functional. With nearly 100% of amputees reporting that they experience discomfort with the wearing of their prosthetic limb, designing an effective interface to the body can significantly affect quality of life and future health outcomes. Active research in medical imaging and biomechanical tissue modeling of residual limbs has led to significant advances in computer aided prosthetic socket design, demonstrating an interest in moving toward more quantifiable processes that are still patient-specific. In our work, medical ultrasonography is being pursued to acquire data that may quantify and improve the design process and fabrication of prosthetic sockets while greatly reducing cost compared to an MRI-based framework. This paper presents a prototype limb imaging system that uses a medical ultrasound probe, mounted to a mechanical positioning system and submerged in a water bath. The limb imaging is combined with three-dimensional optical imaging for motion compensation. Images are collected circumferentially around the limb and combined into cross-sectional axial image slices, resulting in a compound image that shows tissue distributions and anatomical boundaries similar to magnetic resonance imaging. In this paper we provide a progress update on our system development, along with preliminary results as we move toward full volumetric imaging of residual limbs for prosthetic socket design. This demonstrates a novel multi-modal approach to residual limb imaging.
BibTeX
@inproceedings{ranger2016d,
  author = {Bryan J. Ranger and Micha Feigin and Xiang Zhang and Al Mireault and Ramesh Raskar and Hugh Herr and Brian Anthony},
  title = {{3D optical imagery for motion compensation in a limb ultrasound system}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2016},
  doi = {10.1117/12.2218386},
}
Figure