SVoRT: Iterative Transformer for Slice-to-Volume Registration in Fetal Brain MRI

Junshen Xu, Daniel Moyer, Patricia Ellen Grant, Polina Golland, Juan Eugenio Iglesias, Elfar Adalsteinsson

Abstract

Slice-to-Volume Registration (SVR) is a critical step in reconstructing high-quality 3D fetal brain MRI from motion-corrupted 2D slices. Traditional optimization-based methods are computationally expensive and sensitive to initial conditions. This paper introduces SVoRT (Slice-to-Volume Registration Transformer), a novel deep learning framework that treats stacks of slices as a sequence. By leveraging the self-attention mechanism of Transformers and an iterative "predict-and-refine" loop, SVoRT achieves significantly higher registration accuracy than previous CNN-based methods in under a second, demonstrating robust generalization to real-world clinical data.

Introduction

Capturing high-quality 3D MRIs of a fetus is notoriously difficult due to unpredictable fetal motion. To compensate, clinicians typically acquire multiple fast 2D "stacks" of images. However, motion between and during slice acquisitions leads to significant misalignment, often described as a "shuffled deck of cards" in 3D space. Slice-to-Volume Registration (SVR) mathematically maps each 2D slice back to its correct position. While deep learning attempts have been made, most previous CNN-based models process slices independently, ignoring the vital spatial context provided by neighboring slices within a stack.

Methodology

SVoRT addresses the limitations of independent slice processing by utilizing a Transformer architecture to model the registration problem as a sequence-to-sequence task.

Architecture and Sequence Modeling

Instead of looking at one slice at a time, SVoRT treats all slices from multiple stacks as a single sequence. The self-attention mechanism allows the model to use information from feature-rich slices to help localize more ambiguous ones, such as those at the very edge of the brain. The model incorporates positional embeddings that encode stack indices, slice indices, and current estimated 3D transformations to help the Transformer understand the 3D spatial layout.

SVoRT Architecture: (a) The iterative transformation update and volume estimation loop. (b) Detailed view of the Transformer encoder and feature extraction pipeline.

Iterative Refinement and Volume Estimation

The model operates through $K$ iterations of a feedback loop. In each step, it predicts a transformation, estimates a 3D volume based on that prediction, and then uses that volume to refine the next transformation estimate. This process includes a differentiable submodule that predicts weights for each slice to identify and ignore outliers, such as slices with extreme motion artifacts. A Conjugate Gradient (CG) solver is integrated to ensure the entire system is end-to-end trainable.

Results

The model was trained on synthetic motion data derived from the FeTA (Fetal Tissue Annotation) dataset and validated on real-world clinical HASTE sequences from Boston Children’s Hospital.

Quantitative Performance

On synthetic data, SVoRT achieved a mean Euclidean Distance (ED) error of 4.35mm, a massive improvement over previous state-of-the-art baselines like SVRnet (~12.8mm) and PlaneInVol (~12.5mm). Notably, while CNN-based methods saw error spikes for slices far from the brain center, SVoRT maintained consistent accuracy across the entire volume.

Qualitative Comparison and Speed

Visual results demonstrate that SVoRT provides a much cleaner initialization for 3D reconstruction, effectively eliminating the "shattered" look common in motion-heavy fetal scans. Furthermore, the average inference time for a full subject is approximately 0.8 seconds, making it orders of magnitude faster than traditional optimization-based SVR algorithms.

Qualitative comparison of SVoRT against SVRnet and PlaneInVol on synthetic cases. SVoRT (third column) closely matches the Reference (fourth column) in axial, coronal, and sagittal views.

Discussion and Conclusion

SVoRT represents a significant advancement in fetal MRI registration by leveraging the geometric logic of brain stacks through Transformers. Its ability to generalize from synthetic training to real clinical scans suggests it learns robust anatomical features. While currently presented as a high-quality initialization for further sub-millimeter refinement, SVoRT paves the way for future end-to-end reconstruction systems that could bypass separate optimization stages entirely.

Summary updated

BibTeX
@inproceedings{xu2022svort,
  author = {Junshen Xu and Daniel Moyer and Patricia Ellen Grant and Polina Golland and Juan Eugenio Iglesias and Elfar Adalsteinsson},
  title = {{SVoRT: Iterative Transformer for Slice-to-Volume Registration in Fetal Brain MRI}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2022},
  doi = {10.1007/978-3-031-16446-0\_1},
}

Loading related content...

Figure