Reconstruction-free segmentation from undersampled k-space using transformers

Yundi Zhang, Nil Stolt-Ansó, Jiazhen Pan, Wenqi Huang, Kerstin Hammernik, Daniel Rueckert

TL;DR

Researchers have developed a Transformer-based model called DiSK that skips the traditional MRI image reconstruction step entirely. By mapping raw, undersampled k-space (frequency) data directly to segmentation masks, the model maintains high accuracy (Dice ~0.90) even at extreme 64x acceleration factors where traditional image-based methods fail.

What Problem Does This Paper Address?

In clinical Magnetic Resonance Imaging (MRI), there is a constant tug-of-war between image quality and scan time. To speed up scans, technicians "undersample" the data—meaning they collect fewer measurements in k-space (the raw frequency domain where MRI data lives).

Standard practice follows a linear pipeline:

  1. Collect k-space data.
  2. Reconstruct an image (using Fourier Transforms or AI).
  3. Segment the image (labeling structures like the heart wall).

The bottleneck is that at high "acceleration factors" (very little data), the reconstruction step produces blurry, noisy images. When the reconstruction is poor, the subsequent segmentation usually fails. The authors of this paper ask a provocative question: Do we actually need the intermediate image? If our end goal is a clinical metric (like heart volume), can we extract that information directly from the raw frequency data?

Methods

The authors introduce DiSK (Direct Segmentation from K-space), an architecture based on the Perceiver Transformer.

  • The Encoder: Traditional Transformers struggle with MRI data because k-space contains hundreds of thousands of points, and standard self-attention complexity grows quadratically ($O(N^2)$). DiSK uses "cross-attention" blocks to project these massive k-space samples into a small, fixed-size "bottleneck" of latent vectors. This makes the computation manageable while allowing the model to attend to global correlations across the entire frequency spectrum.
  • The Decoder: Instead of outputting a grid of pixels, the decoder is coordinate-based. It takes the latent features from the encoder and "queries" specific image-domain coordinates $(x, y, t)$ to determine the probability that a specific point belongs to a certain tissue class.
  • Key Innovation: Unlike Convolutional Neural Networks (CNNs), which look at local neighborhoods, this Transformer approach treats k-space as an unordered set of points. This makes it naturally robust to different sampling patterns and varying levels of sparsity.

Datasets

The study utilized the UK Biobank dataset, specifically:

  • 1,200 mid-ventricular slices of cardiac short-axis scans.
  • The data was split into 1,000 for training, 100 for validation, and 100 for testing.
  • Synthetic Undersampling: To simulate real-world fast scans, the researchers applied Cartesian undersampling masks and Gaussian $B_0$ variations to the raw data at acceleration factors of 4x, 8x, 16x, 32x, and 64x.

Key Insights and Ideas

  • Segmentation vs. Reconstruction: The core "aha moment" is the realization that reconstruction is a harder task than segmentation. Reconstruction requires recovering fine-grained texture and pixel-level precision. Segmentation only requires identifying boundaries and shapes. By skipping reconstruction, the model doesn't waste its "computational budget" trying to fix image noise that doesn't matter for the final mask.
  • Global Context is King: In k-space, a single data point contains information about the entire image. Transformers are uniquely suited for this because their attention mechanism is inherently global, whereas CNNs are limited by their local receptive fields.
  • Robustness to Sparsity: The model's performance is remarkably flat. While traditional methods get significantly worse as you remove data, DiSK remains stable, suggesting it is highly efficient at extracting the "signal" from the "noise."

Key Results

The researchers compared DiSK against two image-based baselines: Syn-Net (a U-Net trained on noisy reconstructions) and LI-Net (an autoencoder-based approach).

  • Dice Score (Higher is better): DiSK achieved a Dice score of approximately 0.902 across all acceleration factors (4x through 64x). In contrast, the baselines started lower (~0.75–0.80) and degraded as acceleration increased.
  • Hausdorff Distance (Lower is better): DiSK showed significantly lower boundary errors (~4.5–4.9 mm) compared to the baselines (~6.5–7.5 mm).
  • Visual Fidelity: Qualitatively, the segmentations from DiSK remained sharp and anatomically plausible even at 64x acceleration, a level where the "zero-filled" k-space (the raw input) looks like unrecognizable static to the human eye.

Limitations and Future Work

  • Spatial Prior Bias: The authors note that in cardiac short-axis scans, the heart is usually in the same general location. There is a risk the model might be "memorizing" the general shape and location of a heart rather than truly "seeing" it in the k-space.
  • Synthetic Data: The study used synthetic undersampling. Real-world MRI scans involve complex hardware artifacts and motion that might challenge the model's performance.
  • Generalization: The current work focuses on 2D+time slices of the heart. Future work will need to determine if this "reconstruction-free" approach scales to complex 3D volumes or other organs where anatomical variability is higher.

Summary updated

Abstract

Motivation: High acceleration factors place a limit on MRI image reconstruction. This limit is extended to segmentation models when treating these as subsequent independent processes. Goal(s): Our goal is to produce segmentations directly from sparse k-space measurements without the need for intermediate image reconstruction. Approach: We employ a transformer architecture to encode global k-space information into latent features. The produced latent vectors condition queried coordinates during decoding to generate segmentation class probabilities. Results: The model is able to produce better segmentations across high acceleration factors than image-based segmentation baselines. Impact: Cardiac segmentation directly from undersampled k-space samples circumvents the need for an intermediate image reconstruction step. This allows the potential to assess myocardial structure and function on higher acceleration factors than methods that rely on images as input.
BibTeX
@inproceedings{zhang2024reconstruction,
  author = {Yundi Zhang and Nil Stolt-Ansó and Jiazhen Pan and Wenqi Huang and Kerstin Hammernik and Daniel Rueckert},
  title = {{Reconstruction-free segmentation from undersampled k-space using transformers}},
  booktitle = {Proceedings on CD-ROM - International Society for Magnetic Resonance in Medicine. Scientific Meeting and Exhibition/Proceedings of the International Society for Magnetic Resonance in Medicine, Scientific Meeting and Exhibition},
  year = {2024},
  doi = {10.58530/2024/0656},
}

Loading related content...

Figure