A four-step validation protocol for radiomic survival analysis in small cohorts: a body composition case study

Mariela Talavera-Muñoz, Cecilia Diana-Albelda, Miguel Antonio Sampedro-Nuñez, Victor Rodriguez-Laval, Jesus Bescos, Álvaro García-MartínarXiv 2026 · arXiv

TL;DR

This paper introduces a rigorous four-step validation protocol designed to prevent overfitting and quantify uncertainty when using high-dimensional radiomics data in small clinical cohorts. By applying this to body composition analysis in neuroendocrine tumor patients, the authors demonstrate how to distinguish genuine prognostic signals from statistical noise.

What Problem Does This Paper Address?

In the world of medical imaging, radiomics—the extraction of hundreds of quantitative features from scans—promises to uncover "hidden" biomarkers. However, when applied to rare diseases like gastroenteropancreatic neuroendocrine tumors (GEP-NET), researchers often face a "small data" problem.

With only a few dozen patients but hundreds of features, machine learning models are incredibly prone to overfitting: they "memorize" the noise in a specific dataset rather than learning a generalizable biological signal. Standard validation techniques often yield overly optimistic results that fail to replicate. This paper tackles the lack of a standardized, conservative framework for validating these high-dimensional survival models in small cohorts.

Methods

The authors propose a "sanity-check" protocol consisting of four distinct controls:

  1. FDR-Corrected Feature Selection: Using False Discovery Rate (FDR) correction to filter out features that might appear significant purely by chance due to multiple comparisons.
  2. Dual-Path Comparison: Comparing two different strategies for handling high dimensionality—selecting a few features upfront (Path A) versus using all features with "embedded" regularization like LASSO or Ridge (Path B).
  3. Repeated Nested Cross-Validation (CV): Instead of a single train/test split, they use nested loops to tune and evaluate models across multiple random "seeds." This quantifies how much the results depend on the "luck of the draw" in splitting the data.
  4. Negative Control: The ultimate "BS detector." They run their entire pipeline on pure Gaussian noise instead of real patient data to see if the system can "find" a signal where none exists.
The four-step validation protocol designed to ensure model robustness.

Datasets

The study utilized a single-center, retrospective cohort of 78 GEP-NET patients from Hospital Universitario de La Princesa in Madrid.

  • Imaging: Baseline diagnostic CT scans.
  • Features: 535 radiomic features (shape, intensity, and texture) extracted from five body compartments at the L3 vertebral level: skeletal muscle, subcutaneous fat, visceral fat, intramuscular fat, and a manual ROI.
  • Ground Truth: Overall survival time and event status (death).
Figure 1: Workflow for radiomic feature extraction. A standardized volume consisting of ten central slices at the L3 vertebral level is selected from the abdominal CT scan. Automatic segmentations are refined by clinical experts to delineate four body composition compartments (skeletal muscle, subcutaneous fat, visceral fat, and intramuscular fat), alongside a fifth compartment, a spherical region of interest (ROI) placed manually in the paraspinal muscles (manual ROI). Finally, 107 radiomic features, encompassing shape, intensity and texture, are extracted from each of the five compartments per patient.

Key Insights and Ideas

  • The "Luck of the Split" is Real: In small cohorts, a single 70/30 train/test split is highly unreliable. The authors found that performance estimates could swing wildly depending on the random seed, emphasizing the need for repeated nested CV.
  • Noise as a Benchmark: The negative control revealed that even with pure noise, models can sometimes achieve a C-index slightly above 0.5. This provides a necessary "floor" for interpreting real results.
  • Beyond Muscle Mass: While clinical focus is often on sarcopenia (muscle loss), this study found that features of subcutaneous fat and muscle texture (compactness) were also strong prognostic candidates.
  • No "Silver Bullet" Model: No single survival model (Cox, Random Forest, SVM, etc.) consistently outperformed the others, suggesting that in small cohorts, the validation protocol is more important than the specific algorithm choice.

Key Results

  • Prognostic Signal: The radiomic models achieved a Harrell’s concordance index (C-index) that exceeded chance by 0.07–0.15. While modest, this signal was clearly absent in the noise-based negative control.
  • Stability: Repeated nested CV reduced the variability of performance estimates by a median of 41% for the C-index compared to single-split evaluations.
  • Biological Links: Features like "Sphericity" in skeletal muscle (indicating compactness) and "GLNU" in subcutaneous fat (linked to tissue extent) were frequently selected as indicators of better prognosis.
Kaplan-Meier survival curves showing the overall survival of the cohort and stratification by age.

Limitations and Future Work

The authors acknowledge that their study is limited by its single-center nature and relatively small sample size, which prevents definitive clinical conclusions. They emphasize that their findings are "hypothesis-generating."

Future work should focus on external validation using multi-center datasets to see if these radiomic signatures hold up across different scanners and protocols. Additionally, incorporating longitudinal data (how body composition changes over time during treatment) could significantly boost the predictive power of these models.

Summary updated

Abstract

Gastroenteropancreatic neuroendocrine tumors (GEP-NET) lack prognostic biomarkers, and computed tomography (CT)-based body composition radiomics at the third lumbar vertebra (L3) is a promising candidate. Small cohorts make high-dimensional radiomic models prone to overfitting and single-split evaluations variable and optimistic. This methodological study proposes a four-step validation protocol for radiomic survival analysis in small cohorts: (1) false discovery rate (FDR)-corrected feature selection, (2) a dual-path comparison of prior feature selection versus embedded regularization, (3) repeated nested cross-validation (CV) and (4) a negative control with Gaussian noise features. In 78 GEP-NET patients, 535 radiomic features from five L3 compartments were used to evaluate five survival models with Harrell's concordance index (C-index), time-dependent area under the curve (AUC) and integrated Brier score (IBS). Step 1 selected at least one feature in 18% of outer folds under noise (nominal level 20%) versus 84% with radiomic features. The dual-path comparison showed no consistent difference between strategies. Repeated nested CV reduced variability across seeds (41% median reduction in C-index standard deviation), but not enough to distinguish models. The nested CV C-index exceeded chance by 0.07-0.15 with radiomic features but not clearly under noise (0.40-0.42 with prior feature selection, 0.47-0.55 with embedded regularization). Frequently selected features tentatively linked greater subcutaneous fat, skeletal muscle compactness and texture to better prognosis. These findings suggest a modest prognostic signal in L3 radiomics, with considerable uncertainty that the protocol quantifies. The protocol may transfer to other small-cohort radiomic survival studies. Code: https://github.com/marielatalaveram/radiomics-survival-validation-protocol.
BibTeX
@misc{talaveramunoz2026four,
  author = {Mariela Talavera-Muñoz and Cecilia Diana-Albelda and Miguel Antonio Sampedro-Nuñez and Victor Rodriguez-Laval and Jesus Bescos and Álvaro García-Martín},
  title = {{A four-step validation protocol for radiomic survival analysis in small cohorts: a body composition case study}},
  year = {2026},
  eprint = {2611.12682},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12682},
}
Figure