Abstract
This paper presents a deep-learning model for deformable registration of ultrasound images at online rates, which we call U-RAFT. As its name suggests, U-RAFT is based on RAFT, a convolutional neural network for estimating optical flow. U-RAFT, however, can be trained in an unsupervised manner and can generate synthetic images for training vessel segmentation models. We propose and compare the registration quality of different loss functions for training U-RAFT. We also show how our approach, together with a robot performing force-controlled scans, can be used to generate synthetic deformed images to significantly expand the size of a femoral vessel segmentation training dataset without the need for additional manual labeling. We validate our approach on both a silicone human tissue phantom as well as on in-vivo porcine images. We show that U-RAFT generates synthetic ultrasound images with 98% and 81% structural similarity index measure (SSIM) to the real ultrasound images for the phantom and porcine datasets, respectively. We also demonstrate that synthetic deformed images from U-RAFT can be used as a data augmentation technique for vessel segmentation models to improve intersection-over-union (IoU) segmentation performance.
Introduction
Vascular access—the process of inserting catheters into veins or arteries—is a foundational medical procedure that requires significant clinical expertise. While robotic systems offer the potential to automate and standardize these procedures, they face a significant technical hurdle: ultrasound image deformation. To maintain acoustic coupling and signal clarity, an ultrasound probe must apply pressure to the skin, which inevitably compresses and shifts the underlying tissues and vessels.
Training deep learning models to accurately segment these vessels requires massive amounts of manually labeled data across various states of compression. However, standard data augmentation techniques, such as random geometric warping, often produce "unphysical" images that do not reflect actual tissue behavior. Furthermore, traditional image registration methods are often too computationally intensive for real-time use or require ground-truth labels that are difficult to obtain for internal anatomy. This paper introduces U-RAFT, an unsupervised framework designed to solve these challenges by performing real-time deformable registration and generating physically realistic synthetic data.
Methodology: The U-RAFT Framework
Architecture and Unsupervised Learning
U-RAFT (Unsupervised Recurrent All-pairs Field Transforms) adapts the state-of-the-art RAFT architecture, originally designed for optical flow in high-resolution RGB video, for the specific constraints of ultrasound imaging. Unlike supervised models, U-RAFT utilizes a Spatial Transformer Network (STN). The model takes a reference image, predicts a deformation field, warps the image, and compares the result to a target deformed image. This allows the network to learn the underlying physics of tissue compression without requiring pre-labeled deformation maps.
Feature-Aware Cyclic Loss
A primary challenge in ultrasound registration is "speckle" noise and low contrast. To address this, the authors developed a novel Feature-Aware Cyclic Loss ($L_{fa-cyclic-us}$). This loss function incorporates two critical components:
- Cyclic Consistency: Ensuring that a transformation from Image A to Image B can be inverted back to Image A accurately, which enforces spatial coherence.
- SIFT Features: By integrating Scale-Invariant Feature Transform (SIFT) descriptors, the model prioritizes the alignment of distinct anatomical landmarks (like vessel walls) over background noise, ensuring that the registration is anchored to actual structures.
Physics-Based Data Augmentation
By utilizing a UR3e robot arm equipped with a force sensor, the researchers captured ultrasound images at specific force levels (e.g., 2N and 10N). U-RAFT can then linearly interpolate between these two deformation fields to generate realistic synthetic images at intermediate force levels (4N, 6N, 8N). Unlike random stretching, these synthetic images are "physically realistic" because they are derived from observed tissue deformation patterns.
Experimental Results
The model was validated using two distinct datasets: a Blue-gel Dataset (a human tissue phantom) and a Live-pig Dataset (in-vivo porcine femoral images). The latter represents the "small data" challenge typical in medical imaging, with only approximately 70 images per force level.
Registration and Realism
U-RAFT demonstrated high fidelity in generating synthetic images. The synthetic outputs achieved a Structural Similarity Index Measure (SSIM) of 98% for the phantom dataset and 81% for the more complex live porcine dataset. This suggests that the model effectively captures the visual characteristics of compressed tissue.
Segmentation Performance
The primary goal of U-RAFT is to improve vessel segmentation, particularly in difficult scenarios like vessel collapse under high pressure. By augmenting the training sets with U-RAFT’s synthetic data, the Intersection-over-Union (IoU) scores improved significantly:
- Blue-gel: IoU increased from 0.86 to 0.89.
- Live-pig: IoU increased from 0.56 to 0.62.
Crucially, the model operates at approximately 33 Hz, meeting the requirements for real-time robotic guidance during clinical procedures.
Discussion and Future Work
U-RAFT successfully addresses the lack of labeled data for deformed ultrasound images by providing a physically grounded augmentation strategy. One of the most significant outcomes was the improved segmentation of nearly collapsed veins—a scenario where standard models often fail due to a lack of training examples.
However, limitations remain. The model's performance slightly decreases during "vessel decollapsing" (the transition from high to low pressure), suggesting that linear interpolation may not fully capture the non-linear viscoelastic properties of certain biological tissues. Future research will focus on integrating formal physics-based models into the recurrent loop and combining U-RAFT with other spatial augmentation techniques to further enhance robustness in live clinical environments.
Summary updated