Joint Motion Correction and Super Resolution for Cardiac Segmentation via Latent Optimisation

Shuo Wang, Chen Qin, Nicoló Savioli, Chen Chen, Declan P. O'Regan, Stuart A. Cook, Yike Guo, Daniel Rueckert, Wenjia Bai

TL;DR

This paper introduces a novel framework that simultaneously corrects motion artifacts and enhances the resolution of cardiac MRI segmentations. By optimizing in a learned latent space, the method achieves state-of-the-art performance without requiring paired low-resolution/high-resolution training data, making it more robust and generalizable.

What Problem Does This Paper Address?

Accurate 3D segmentation of the heart from cardiac magnetic resonance (CMR) imaging is crucial for understanding its structure and function. However, clinical CMR acquisition protocols typically capture stacks of 2D slices due to long breath-hold requirements, leading to low-resolution images with anisotropic resolution (lower resolution along the slice direction) and motion artifacts. These degradations make precise segmentation challenging. Existing super-resolution methods often rely on paired low-resolution (LR) and high-resolution (HR) data for training, which can be difficult to obtain in real-world scenarios and may lead to poor performance when applied to data from different domains or with different degradation patterns. This paper aims to address these limitations by developing a method that jointly handles motion correction and super-resolution in a more flexible and robust manner.

Methods

The proposed framework, termed "latent optimisation" (LO), operates in two main stages:

  1. Learning the High-Resolution Manifold: A generative model (specifically, a β-VAE or DCGAN) is trained to learn the distribution of plausible high-resolution cardiac segmentations. This model acts as an anatomical prior, ensuring that generated segmentations are anatomically realistic. It maps a low-dimensional latent space ($z$) to the high-dimensional space of HR segmentations ($\Sigma_{HR}$).
  2. Latent Optimisation: Given a low-resolution segmentation ($S_{LR}$), the framework iteratively optimizes a latent variable ($z$) and motion parameters ($d$) to find the HR segmentation ($SSR$) that best explains the input. This optimization is guided by a multi-view loss function that enforces consistency between the degraded HR segmentation generated by the model and the input $S_{LR}$. The loss function can incorporate both short-axis (SA) and, optionally, long-axis (LA) views. The optimization is performed in the latent space, which is significantly lower dimensional than the HR segmentation space, making the process more efficient and stable. The motion correction is handled by a differentiable inter-slice motion layer, and super-resolution is achieved by the generative model.

Datasets

The paper uses two datasets:

  • HR dataset: A research cohort of CMR images from 1,200 healthy volunteers with a spatial resolution of 1.25x1.25x2 mm, considered "high-resolution." Segmentations were performed using an atlas-based method with manual correction. This dataset is used for simulating degradations.
  • UK Biobank dataset: A low-resolution dataset containing short-axis image stacks from 200 subjects with a spatial resolution of 1.8x1.8x10 mm. This dataset also includes a 4-chamber long-axis view. Segmentations were performed automatically using a fully-convolutional network. This dataset is used for real-world evaluation.

The authors also created simulated datasets by applying various degradation patterns (no-motion, normal-motion, severe-motion) to the HR dataset to rigorously test the method's robustness.

Key Insights and Ideas

  • Joint Optimization is Key: Treating motion correction and super-resolution as a single, jointly optimized problem, rather than sequential steps, leads to better overall results.
  • Latent Space for Robustness: Optimizing in a learned latent space, guided by a generative model, provides a strong anatomical prior. This makes the method less sensitive to variations in input data and degradation patterns compared to purely data-driven supervised methods.
  • No Paired Data Needed: The framework bypasses the need for paired LR-HR training data, which is a significant practical advantage for clinical applications where such data is scarce or difficult to acquire.
  • Multi-view Consistency: Incorporating information from different views (e.g., short-axis and long-axis) further improves segmentation accuracy and robustness.
  • Segmentation as Input: Working with segmentations instead of raw images offers convenience and potentially better generalizability across different imaging modalities, devices, or contrast agents.

Key Results

  • Simulated Degradation: The proposed LO method achieved state-of-the-art performance on simulated datasets, outperforming traditional methods (NN, SBI) and even supervised deep learning methods (EDSR) under severe motion degradation. For instance, under severe motion, the LO method achieved Dice scores of 0.92±0.02 for LV, 0.75±0.08 for MYO, and 0.91±0.02 for RV, outperforming EDSR (0.89±0.02, 0.70±0.07, 0.89±0.03 respectively).
  • Generalization: The LO method demonstrated significantly better generalization ability than EDSR. When EDSR was trained on one degradation pattern and tested on another, its performance dropped considerably. In contrast, the LO method maintained high performance across different degradation patterns, showing a generalization improvement of over 4% in Dice score.
  • Real-World Performance: On the UK Biobank dataset, the LO method consistently outperformed other models for all cardiac structures, with the LO-multi-view variant showing further improvements.

Limitations and Future Work

  • Computational Cost: While the latent space optimization is more efficient than optimizing in the full HR space, it still involves an iterative optimization process, which can be computationally more intensive than a single forward pass of a trained network.
  • Generative Model Choice: The performance of the framework is dependent on the quality of the learned manifold by the generative model. The paper explored β-VAE and DCGAN, with β-VAE showing better results. Further exploration of different generative architectures could be beneficial.
  • Future Work: The authors suggest that the framework's flexibility could be extended to other medical imaging modalities and tasks. The ability to incorporate multi-view information and its robustness to domain shifts are highlighted as key advantages for future applications.

Summary updated

BibTeX
@inproceedings{wang2021joint,
  author = {Shuo Wang and Chen Qin and Nicoló Savioli and Chen Chen and Declan P. O'Regan and Stuart A. Cook and Yike Guo and Daniel Rueckert and Wenjia Bai},
  title = {{Joint Motion Correction and Super Resolution for Cardiac Segmentation via Latent Optimisation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  doi = {10.1007/978-3-030-87199-4\_2},
}

Loading related content...

Figure