VINNA for neonates: Orientation independence through latent augmentations

Leonie Henschel, David Kügler, Lilla Zöllei, Martin Reuter

TL;DR

This paper introduces VINNA, a novel deep learning framework for segmenting neonatal brain MRIs. VINNA tackles the challenges of varying head positions and resolutions by integrating a 4-DOF transform module directly into the network, performing "internal augmentations" without image interpolation, leading to significantly improved accuracy and robustness compared to existing methods.

What Problem Does This Paper Address?

Accurate segmentation of neonatal brain MRIs is crucial for understanding brain development and detecting early signs of disorders. However, this task is hampered by several factors:

  • Limited Data: High-quality, annotated neonatal brain MRI datasets are scarce.
  • Variability: Newborns exhibit significant variations in head positioning within the scanner due to sleep and smaller anatomy.
  • Inconsistent Protocols: There's no standardized acquisition protocol, leading to a wide range of resolutions (0.5-3.0 mm) across studies.

Traditional methods often rely on time-consuming spatial registration and resampling to a common resolution, which can lead to loss of information and reduced accuracy, especially with label interpolation. Deep learning approaches typically use "external augmentation" (applying transformations to images and labels outside the network), which also requires resampling and can degrade label accuracy. VINNA aims to overcome these limitations by performing augmentations internally within the network.

Methods

The core innovation of VINNA is its network-integrated 4-DOF transform module. This module allows for spatial augmentations (rotation, translation, and scaling) to be applied internally to the network's feature maps, rather than externally to the raw images and labels.

  • Resolution Independence (VINN): Building on their previous work (VINN), the framework inherently handles varying resolutions without requiring resampling.
  • 4-DOF Transform Module: This module is integrated into the network architecture. It uses a transformation matrix that encodes rotation, translation, and scaling. These parameters are randomly sampled during training, effectively creating "internal augmentations."
  • Avoiding Interpolation: Crucially, this internal augmentation avoids the lossy interpolation of discrete label maps that occurs with external augmentation. Instead, it operates on continuous feature maps.
  • Network Architecture: VINNA uses a UNet-like architecture with competitive dense blocks (CDBs) and incorporates the 4-DOF module in the encoder and decoder stages.

Datasets

  • Training & Validation: The Developing Human Connectome Project (dHCP) dataset was used, comprising T1w and T2w MRIs from newborns aged 24-44 weeks post-menstrual age.
  • Testing:
    • The dHCP dataset was also used as an independent test set.
  • The Melbourne Children's Regional Infant Brain (M-CRIB) atlas cohort was used for external validation. This dataset consists of T2w MRIs from term-born neonates.

Key Insights and Ideas

  • Internal Augmentation is Superior: Shifting spatial augmentation from the image space (external) to the feature map space (internal) within the network significantly reduces information loss and improves segmentation accuracy, especially for challenging structures like subcortical regions.
  • Resolution Independence is Key: The ability to process images at their native resolution without resampling is fundamental for maintaining accuracy, particularly in the context of varying acquisition protocols.
  • Newborns are Different: The inherent variability in head positioning and resolution in neonatal datasets necessitates specialized approaches beyond standard adult segmentation techniques.
  • 4-DOF Module is Generalizable: While applied to neonates here, the concept of a network-integrated transform module could be extended to other tasks and degrees of freedom.

Key Results

  • Outperforms State-of-the-Art: VINNA significantly outperformed existing methods, including external augmentation approaches (CNN* + exA, VINN + exA) and established pipelines like nnUNet, iBEAT, and infantFS.
  • Improved Accuracy: VINNA achieved the highest Dice Similarity Coefficient (DSC) and lowest Average Surface Distance (ASD) across various resolutions (0.5-1.0 mm) and structures (cortex, white matter, subcortical). For example, on T2w images at 0.5 mm, VINNA achieved an average DSC of 95.33% and ASD of 0.102 mm.
  • Robustness to Resolution: The framework maintained high accuracy across different resolutions, demonstrating true resolution independence.
  • Handles Head Variations: VINNA effectively addressed the increased variability in head positions common in newborn datasets.
  • Qualitative Improvements: VINNA captured finer structural details and produced less smoothed segmentations compared to other methods, aligning better with ground truth.

Limitations and Future Work

  • Limited Extrapolation: While VINNA performs well across tested resolutions and age groups within the dHCP cohort, generalization to datasets with significantly different T1w intensity profiles or acquisition parameters is not guaranteed and requires dedicated validation.
  • T1w vs. T2w Performance: Performance on T1w images was generally lower than on T2w images, likely due to the lower contrast and less mature myelination in T1w scans of newborns, which can lead to ambiguous tissue boundaries.
  • Future Directions:
    • Exploring the generalization of the 4-DOF module to higher degrees of freedom (e.g., 9-DOF for 3D) and other tasks like classification.
  • Investigating generative models (e.g., SynthSeg-like intensity augmentation) to improve generalization across varying contrast and age ranges.
  • Addressing the challenge of limited, non-uniform segmentation labels by exploring semi-supervised or unsupervised learning approaches.
  • Developing unified segmentation definitions across different atlases, modalities, and age ranges.
  • Adapting the infantFS surface pipeline to VINNA4neonates predictions for enhanced structural segmentation.

Summary updated

Abstract

A robust, fast, and accurate segmentation of neonatal brain images is highly desired to better understand and detect changes during development and disease, specifically considering the rise in imaging studies for this cohort. Yet, the limited availability of ground truth datasets, lack of standardized acquisition protocols, and wide variations of head positioning in the scanner pose challenges for method development. A few automated image analysis pipelines exist for newborn brain Magnetic Resonance Image (MRI) segmentation, but they often rely on time-consuming non-linear spatial registration procedures and require resampling to a common resolution, subject to loss of information due to interpolation and down-sampling. Without registration and image resampling, variations with respect to head positions and voxel resolutions have to be addressed differently. In deep learning, external augmentations such as rotation, translation, and scaling are traditionally used to artificially expand the representation of spatial variability, which subsequently increases both the training dataset size and robustness. However, these transformations in the image space still require resampling, reducing accuracy specifically in the context of label interpolation. We recently introduced the concept of resolution-independence with the Voxel-size Independent Neural Network framework, VINN. Here, we extend this concept by additionally shifting all rigid-transforms into the network architecture with a four degree of freedom (4-DOF) transform module, enabling resolution-aware internal augmentations (VINNA) for deep learning. In this work, we show that VINNA (i) significantly outperforms state-of-the-art external augmentation approaches, (ii) effectively addresses the head variations present specifically in newborn datasets, and (iii) retains high segmentation accuracy across a range of resolutions (0.5-1.0 mm). Furthermore, the 4-DOF transform module together with internal augmentations is a powerful, general approach to implement spatial augmentation without requiring image or label interpolation. The specific network application to newborns will be made publicly available as VINNA4neonates.
BibTeX
@inproceedings{henschel2024vinna,
  author = {Leonie Henschel and David Kügler and Lilla Zöllei and Martin Reuter},
  title = {{VINNA for neonates: Orientation independence through latent augmentations}},
  booktitle = {Imaging},
  year = {2024},
  doi = {10.1162/imag\_a\_00180},
}
Figure