Abstract
This paper proposes a method for trackerless ultrasound volume reconstruction in the context of minimally invasive surgery. It is based on a Siamese architecture, including a recurrent neural network that leverages the ultrasound image features and the optical flow to estimate the relative position of frames. Our method does not use any additional sensor and was evaluated on ex vivo porcine data. It achieves translation and orientation errors of $0.449 \pm 0.189$ mm and $1.3 \pm 1.5^{\circ}$ respectively for the relative pose estimation. In addition, despite the predominant non-linearity motion in our context, our method achieves a good reconstruction with final and average drift rates of 23.11% and 28.71% respectively. To the best of our knowledge, this is the first work to address volume reconstruction in the context of intravascular ultrasound.
Introduction
In liver surgery, precise identification of internal landmarks such as blood vessels and tumors is critical. While intraoperative ultrasound (IOUS) is standard, its application in laparoscopic procedures is limited by the bulkiness of traditional probes. Intravascular Ultrasound (IVUS) offers a compelling alternative, providing an "inside-out" perspective from within the vena cava. However, transforming 2D IVUS slices into a 3D volume typically requires external tracking sensors (e.g., electromagnetic trackers), which are costly and sensitive to metallic interference in the operating room.
This work introduces a trackerless approach, estimating probe motion directly from the images. Unlike previous methods focused on linear sweeps, this research specifically addresses the complex rotational motions characteristic of IVUS catheters.
Methodology
The proposed system utilizes a Siamese Seq2Vec (Sequence-to-Vector) architecture that integrates visual features with geometric motion priors.
Sparse Optical Flow and Heatmaps
To move beyond raw pixel analysis, the system tracks prominent points across frame sequences using the Lucas-Kanade method. These trajectories are converted into Gaussian heatmaps, providing the network with explicit geometric "hints" about probe movement in an image-like format.
Deep Learning Architecture
The core of the model consists of five blocks of Convolutional LSTMs (ConvLSTM) and Batch Normalization. This recurrent structure is designed to capture temporal dependencies across a sequence of seven frames. The network outputs a 6-degree-of-freedom (6 DoF) vector (three translations and three Euler angles) representing the relative pose between frames.
Loss Function
Training is optimized using a combination of Mean Square Error (MSE) on frame corners and an accumulation loss. The latter is crucial for mitigating "drift"—the compounding error that often degrades trackerless reconstruction over long sequences.
Experimental Results
The method was validated using an ex vivo porcine liver dataset comprising 13,734 clips. Ground truth was established using a trakSTAR electromagnetic tracking system.
| Metric | Result |
|---|---|
| Translation Error | $0.449 \pm 0.189$ mm |
| Orientation Error | $1.3 \pm 1.5^{\circ}$ |
| Final Drift Rate (FDR) | 23.11% |
| Average Drift Rate (ADR) | 28.71% |
The results demonstrate high-precision relative pose estimation. Notably, this purely image-based approach achieves performance parity with state-of-the-art models that rely on additional Inertial Measurement Unit (IMU) sensors.
Discussion and Future Work
While the relative pose estimation is highly accurate, cumulative drift remains a challenge for long-term volumetric consistency. Future research will focus on reducing this drift and extending the model to handle organ deformation. Additionally, the authors aim to register these reconstructed 3D volumes with pre-operative CT scans to enhance intraoperative navigation.
Summary updated