TL;DR
This paper introduces C-SliceGen, a novel deep learning model that harmonizes longitudinal abdominal CT scans by generating consistent slices at a target vertebral level, regardless of the original slice's position. This addresses a critical challenge in analyzing body composition changes over time, enabling more accurate and reliable longitudinal studies.
What Problem Does This Paper Address?
Longitudinal studies using abdominal CT scans are invaluable for understanding how body composition changes with aging and its relation to various health conditions. However, a significant hurdle arises from the positional variation of CT slices acquired at different times. Even within the same individual, scans taken years apart might capture different cross-sections of the abdomen, leading to variations in the visible organs and tissues. This "positional variance" makes it difficult to accurately compare body composition metrics (like visceral fat area) across different time points, hindering the ability to track subtle changes and draw reliable conclusions. Existing methods like image registration are not ideal for this out-of-plane motion problem, and while deep learning generative models show promise, they often lack the control needed to generate slices at a specific, consistent anatomical location.
Methods
The core of this work is the C-SliceGen model, built upon a Variational Autoencoder Generative Adversarial Network (VAEGAN) architecture. This hybrid approach combines the strengths of VAEs (for learning latent representations and reconstruction) and GANs (for generating sharp, realistic images).
Here's a breakdown of the key components:
- Conditional Generation: Unlike standard generative models, C-SliceGen is conditional. It takes an arbitrary axial abdominal CT slice as input and is trained to generate a new slice at a pre-defined target vertebral level. This target slice acts as the "condition" for generation.
- Dual Encoders: The model uses two encoders. One encoder processes the input (arbitrary) slice to capture subject-specific information (like body habitus), and another encoder processes the target slice to learn its structural characteristics.
- VAEGAN Backbone: The VAE component handles the reconstruction of the target slice and learning of latent representations. The GAN component, with its generator and discriminator, ensures the generated slices are realistic and sharp, avoiding the blurriness often associated with VAEs.
- Target Slice Selection: A crucial preprocessing step is selecting a consistent "target slice" for each individual. The paper explores two methods:
- BPR-based method: Uses Body Part Regression scores to find slices with similar anatomical regions.
- Registration-based method: Employs image registration to align slices and identify the most similar one.
- A semi-BPR method is also introduced to improve target slice selection, especially for non-contrast data.
- Loss Function: The model is trained using a combined loss function that includes reconstruction loss, generation loss, and KL-divergence (from VAEs), along with adversarial loss from the GAN component.
Datasets
The researchers utilized several datasets for training and evaluation:
- In-house portal venous CT dataset: 1120 3D CT volumes from Vanderbilt University Medical Center (VUMC). Used for training and validation.
- In-house non-contrast CT dataset: 1488 3D CT volumes. Used to address domain shift issues and for validation.
- MICCAI 2015 Multi-Atlas Abdomen Labeling Challenge (BTCV) dataset: 30 portal venous CT volumes for training and 20 for testing. Used for external validation and fine-tuning.
- Baltimore Longitudinal Study of Aging (BLSA) dataset: A crucial dataset for evaluating longitudinal harmonization. It comprises 4223 2D single-slice non-contrast CT scans from 1033 subjects, collected over multiple visits. This dataset is key for assessing the model's ability to harmonize positional variance in real-world longitudinal data.
Key Insights and Ideas
- Conditional Generation for Anatomical Consistency: The core innovation is using a target slice as a condition to guide the generative process. This allows the model to learn to generate slices at a specific anatomical level, effectively "normalizing" the position across different scans.
- VAEGAN for Sharp and Realistic Images: Combining VAEs and GANs proves effective in generating high-quality images that are both structurally similar to the target and perceptually realistic, overcoming limitations of each individual model.
- Positional Variance is a Major Bottleneck: The BLSA dataset analysis clearly demonstrates how positional variance significantly impacts body composition metrics like visceral fat area, highlighting the need for harmonization.
- Adversarial Regularization Improves Quality: The ablation study shows that adversarial regularization (using GAN) significantly improves the quality of generated images, making them sharper and more realistic compared to purely VAE-based generation.
- Distance Matters, But Not Infinitely: The distance between the input slice and the target slice impacts performance. While very small distances (3-6mm) can lead to data deficiency issues, performance is stable and good within a range of 15-60mm. Training with an "unknown distance" across the entire abdominal region leads to poorer performance.
Key Results
- 3D Datasets: The model achieved good quantitative results (SSIM, PSNR, LPIPS, NML) on 3D datasets, demonstrating its ability to generate realistic and structurally similar target slices. The semi-BPR method showed promise for non-contrast data.
- 2D Single-Slice (BLSA) Harmonization:
- NML: Harmonized images showed higher Normalized Mutual Information (NML) compared to original images, indicating greater similarity between scans of the same subject after harmonization.
- CV: For subjects with significant positional variation (different vertebral levels), the model significantly reduced the Coefficient of Variation (CV) (p < 0.01), demonstrating effective harmonization.
- Visceral Fat Area: Visualizations and quantitative analysis showed that the generated images harmonized organ/tissue differences, leading to reduced variation in visceral fat area across visits.
- Adversarial Regularization: Using a $\beta$ score of 0.01 for adversarial regularization resulted in significantly better generated image quality (sharper, more realistic) compared to $\beta = 0$.
Limitations and Future Work
- Heterogeneous Soft Tissues: A critical limitation is the model's difficulty in accurately predicting and synthesizing highly variable soft tissues like the colon and stomach. Their size and shape are heavily dependent on individual conditions and scan timing, making them challenging to model consistently. Future work could focus on improving the generation of these tissues and preserving their shape and boundary information.
- Introduction of Noise: When input slices are already at similar vertebral levels, the model can sometimes introduce additional noise, leading to larger variance.
- Optimal Data Collection Range: The study suggests that for optimal performance, future longitudinal CT data collection might benefit from focusing on slices within a range of approximately 60 mm (around ±3 vertebral levels) from the target slice.
- Domain Shift: While improved, the domain shift between portal venous and non-contrast phases still presents a challenge, though the use of non-contrast data for training and semi-BPR helps mitigate this.
Summary updated