Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern

Niklas Holzwarth, Melanie Schellenberg, Janek Gröhl, Kris Dreher, Jan-Hinrich Nölke, Alexander Seitel, Minu Dietlinde Tizabi, Beat P. Müller-Stich, Lena Maier-Hein

TL;DR

This paper introduces "Tattoo tomography," a novel, hardware-free method for creating 3D photoacoustic (PA) images from freehand 2D scans. By using a simple optical pattern placed on the skin, the system can accurately track the probe's position and orientation, enabling precise 3D reconstruction without external tracking systems or complex calibration.

What Problem Does This Paper Address?

Photoacoustic Tomography (PAT) is a powerful imaging technique that can visualize both the structure and function of tissues, like blood vessels and oxygenation levels. However, current clinical PAT systems typically use 2D probes, which only capture cross-sectional slices. To get a full 3D picture, researchers have explored various methods, but these often rely on bulky external tracking systems, complex robotic arms, or require extensive application-specific training data. These limitations hinder seamless integration into clinical workflows and can be costly. The core problem this paper addresses is the lack of a simple, accurate, and clinically friendly method for freehand 3D PAT reconstruction.

Methods

The "Tattoo tomography" approach cleverly uses an optical pattern applied to the skin over the area of interest. This pattern is designed to be visible in the PA images but at a different wavelength than the functional imaging. The key steps are:

  1. Pattern Attachment: A simple, reusable optical pattern (initially a trident shape) is attached to the skin. This pattern absorbs light at a specific wavelength (e.g., 750 nm) that is distinct from the imaging wavelength (e.g., 850 nm).
  2. Image Acquisition: Freehand 2D PA images are acquired. Each image captures a portion of the optical pattern along with the underlying tissue.
  3. Pose Estimation: The unique features of the optical pattern within each 2D PA image are used to determine the probe's pose (position and orientation) relative to the pattern's coordinate system. This is done by analyzing the intersection of the probe's imaging plane with the pattern. The method specifically estimates three degrees of freedom (DoF) based on the pattern's geometry, with two additional constraints (orthogonality and approximate flatness) simplifying the process for this prototype.
  4. 3D Volume Compounding: Once the pose of each 2D slice is known in a common coordinate system (defined by the pattern), the slices are transformed and interpolated to create a coherent 3D volume. This process is inspired by existing volume reconstruction algorithms for ultrasound.

Datasets

The paper describes experiments using:

  • Experimental Phantom Data: An N-wire phantom, commonly used for calibrating tracked devices, was employed for quantitative validation of accuracy. This phantom consists of three layers of N-shaped black nylon wires, providing strong broadband absorption.
  • In Vivo Forearm Data: Data was collected from three healthy volunteers, with scans performed on their forearms to assess the precision and reproducibility of the method.

No new datasets were introduced for training; the method is designed to be sensorless and not rely on pre-trained models.

Key Insights and Ideas

  • Leveraging PAT's Dual-Wavelength Capability: The method ingeniously uses different wavelengths for the optical pattern and for functional tissue imaging. This allows the pattern to be clearly visible for pose estimation without interfering with the actual diagnostic information.
  • Simplicity as a Strength: The core idea is remarkably simple: a sticker on the skin provides the necessary reference for 3D reconstruction. This bypasses the need for expensive external tracking hardware, complex calibration procedures, or application-specific training data, making it highly attractive for clinical translation.
  • "Tattoo" as a Coordinate System: The optical pattern effectively acts as a temporary, local coordinate system. By aligning all acquired PA slices to this "tattoo" coordinate system, a consistent 3D reconstruction is achieved.
  • Potential for Higher Accuracy/Precision: Initial results suggest that this pattern-based approach might offer better accuracy and precision than some existing methods that rely on external tracking systems, possibly due to the tattoo system's relative nature and reduced susceptibility to patient motion.

Key Results

  • Feasibility Demonstration: The method successfully reconstructed a logo (IPCAI) from freehand scans, even with careless probe movements, demonstrating its robustness.
  • Phantom Validation (Accuracy): The Tattoo tomography approach achieved an average Fiducial Registration Error (FRE) of 0.63 mm, which was slightly better than the baseline optical tracking method (0.87 mm) when registering reconstructed volumes to a known phantom model.
  • In Vivo Validation (Precision): The mean vessel distance (MVD) for the Tattoo approach was 0.63 mm and 0.80 mm for two volunteers, significantly outperforming the baseline optical tracking method (1.40 mm and 1.06 mm, respectively). This indicates higher reproducibility.

Limitations and Future Work

  • Current 3 DoF Limitation: The current prototype pattern design only allows for the recovery of three degrees of freedom (DoF). This requires the probe to be held approximately orthogonally to the pattern plane.
  • Surface Flatness Requirement: The method currently assumes a relatively flat target surface.
  • Future Work:
    • Developing 3D optical patterns to recover all six DoF, eliminating the need for the orthogonality constraint.
  • Employing image processing techniques to handle non-flat tissue surfaces.
  • Enhancing the optical pattern for multi-modal registration with other imaging modalities like CT or MRI.
  • Further studies are needed to confirm the initial findings, especially regarding the in vivo precision and the influence of patient motion.

Summary updated

Abstract

Purpose: Photoacoustic tomography (PAT) is a novel imaging technique that can spatially resolve both morphological and functional tissue properties, such as the vessel topology and tissue oxygenation. While this capacity makes PAT a promising modality for the diagnosis, treatment and follow-up of various diseases, a current drawback is the limited field-of-view (FoV) provided by the conventionally applied 2D probes. Methods: In this paper, we present a novel approach to 3D reconstruction of PAT data (Tattoo tomography) that does not require an external tracking system and can smoothly be integrated into clinical workflows. It is based on an optical pattern placed on the region of interest prior to image acquisition. This pattern is designed in a way that a tomographic image of it enables the recovery of the probe pose relative to the coordinate system of the pattern. This allows the transformation of a sequence of acquired PA images into one common global coordinate system and thus the consistent 3D reconstruction of PAT imaging data. Results: An initial feasibility study conducted with experimental phantom data and in vivo forearm data indicates that the Tattoo approach is well-suited for 3D reconstruction of PAT data with high accuracy and precision. Conclusion: In contrast to previous approaches to 3D ultrasound (US) or PAT reconstruction, the Tattoo approach neither requires complex external hardware nor training data acquired for a specific application. It could thus become a valuable tool for clinical freehand PAT.
BibTeX
@article{holzwarth2021tattoo,
  author = {Niklas Holzwarth and Melanie Schellenberg and Janek Gröhl and Kris Dreher and Jan-Hinrich Nölke and Alexander Seitel and Minu Dietlinde Tizabi and Beat P. Müller-Stich and Lena Maier-Hein},
  title = {{Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern}},
  journal = {Int. J. Comput. Assist. Radiol. Surg.},
  year = {2021},
  url = {https://openreview.net/forum?id=JzM43ZIoM1},
}
Figure