Automatic size and pose homogenization with spatial transformer network to improve and accelerate pediatric segmentation

Giammarco La Barbera, Pietro Gori, Haithem Boussaid, Bruno Belucci, Alessandro Delmonte, Jeanne Goulin, Sabine Sarnacki, Laurence Rouet, Isabelle Bloch

Introduction

Segmenting organs and tumors in pediatric imaging presents unique challenges compared to adult populations. The primary difficulties stem from extreme anatomical heterogeneity—ranging from newborns to adolescents—and significant variations in pose due to patient movement during scans. Furthermore, large-scale pediatric datasets are scarce, forcing models to rely on computationally expensive data augmentation to achieve invariance to scale and rotation. While transfer learning from adult models is a common strategy, it often underperforms because the relative proportions and developmental stages of organs differ significantly between children and adults.

Methods

The authors propose a novel three-stage sequential pipeline designed to "homogenize" pediatric data, standardizing its appearance before segmentation. This approach reduces the geometric search space, allowing the segmentation network to focus on fine anatomical details.

The proposed three-stage pipeline featuring two STN modules for normalization and cropping followed by a UNet for segmentation.

The architecture consists of three primary modules:

  1. Pose & Size Normalization (STN1): A Spatial Transformer Network (STN) estimates a similarity matrix to align the input image to a reference pediatric template, standardizing pose and scale.
  2. ROI Cropping (STN2): A second differentiable STN automatically identifies and crops a square Region of Interest (ROI) around the target organs. This reduces the memory footprint significantly (e.g., from ~10GB to ~3.5GB for 512x512 images).
  3. Segmentation (nnUNet): A standard nnUNet performs the final segmentation on the "cleaned" input.

Importantly, the transformations are reversible, enabling the final segmentation masks to be mapped back to the original image space.

Datasets and Experimental Setup

The study utilized an internal dataset of 80 pediatric abdominal-visceral CT scans (ages 3–16) with manual annotations for kidneys and renal tumors. For comparison, the KiTS19 adult database (210 patients) was used to evaluate transfer learning performance. The proposed pipeline was compared against a baseline nnUNet using standard "on-the-fly" data augmentation.

Results

The STN-based homogenization approach demonstrated superior performance in both efficiency and accuracy, particularly for challenging renal tumor segmentation.

Comparison of training time and Dice scores between standard nnUNet and the proposed STN-based approach.

Key findings include:

  • Improved Accuracy: For 512x512 images, the STN-based pipeline improved the Dice score for renal tumors from 85.52% to 87.12%.
  • Accelerated Training: The proposed method reduced training time from 33 hours to 25 hours compared to standard data augmentation.
  • Consistency: The model exhibited lower standard deviation in results, indicating greater robustness across the variable pediatric population.
  • Adult-to-Child Gap: Direct inference using adult-trained models on pediatric data yielded poor results (Dice scores ~20%), confirming the necessity of pediatric-specific solutions.

Conclusion

By integrating Spatial Transformer Networks for data homogenization, this research demonstrates that reducing geometric variability upfront is more efficient than relying on brute-force data augmentation. This approach not only improves segmentation accuracy for complex structures like renal tumors but also significantly reduces training time and memory requirements. Future work will explore extending this framework to 3D volumes and streamlining the STN modules into a single component.

Summary updated

BibTeX
@inproceedings{barbera2021automatic,
  author = {Giammarco La Barbera and Pietro Gori and Haithem Boussaid and Bruno Belucci and Alessandro Delmonte and Jeanne Goulin and Sabine Sarnacki and Laurence Rouet and Isabelle Bloch},
  title = {{Automatic size and pose homogenization with spatial transformer network to improve and accelerate pediatric segmentation}},
  booktitle = {ISBI},
  year = {2021},
  doi = {10.1109/isbi48211.2021.9434090},
}
Figure