Artificial Intelligence for Response Assessment in Neuro Oncology (AI-RANO), part 2: recommendations for standardisation, validation, and good clinical practice

Spyridon Bakas, Philipp Kickingereder, Norbert Galldiks, Thomas C. Booth, Hugo J.w.l. Aerts, Wenya Linda Bi, Benedikt Wiestler, Pallavi Tiwari, Sarthak Pati, Ujjwal Baid, Evan Calabrese, Philipp Lohmann, Martha Nowosielski, Rajan Jain, Rivka R. Colen, Marwa Ismail, Ghulam Rasool, Janine Lupo, Hamed Akbari, J. C. Tonn, David R. Macdonald, Michael A. Vogelbaum, Susan M. Chang, Christos Davatzikos, Javier Villanueva-Meyer, Raymond Y. HuangThe Lancet Oncology 2024 · DOI

Introduction

Artificial intelligence (AI) holds immense potential to transform neuro-oncology by providing objective, reproducible assessments of brain tumor response to therapy. However, despite a decade of research, most AI models remain confined to academic settings. This "reproducibility crisis" stems from a lack of standardization across different MRI scanners, surgical practices, and radiological interpretations. The AI-RANO working group addresses these challenges by proposing a comprehensive roadmap for transitioning AI from experimental research to routine clinical practice.

The Challenge of Response Assessment

A primary hurdle in neuro-oncology is distinguishing true tumor progression from pseudoprogression—a treatment-related inflammatory response that mimics growth on MRI. Current clinical decisions often rely on subjective interpretations, which can vary significantly between radiologists. Furthermore, AI models trained at one institution frequently fail when applied elsewhere due to data heterogeneity. To overcome this "Wild West" environment, the field requires rigorous standards for data acquisition, preprocessing, and validation.

A Framework for Good Clinical Practice

This policy review synthesizes expert consensus into a framework for robust AI pipelines in neuro-oncology. Key recommendations include:

1. Standardized Preprocessing and Feature Extraction

To ensure comparability across different imaging sites, the authors recommend using canonical anatomical templates, such as the SRI24 brain atlas, and standardized pipelines like BraTS (Brain Tumor Segmentation). Furthermore, the adoption of the Image Biomarker Standardisation Initiative (IBSI) ensures that radiomic features—mathematical descriptions of tumor texture and shape—are calculated consistently across different software platforms.

2. Federated Learning

To address data privacy concerns while leveraging large datasets, the review highlights Federated Learning. This "private-by-design" approach allows AI algorithms to be trained across multiple institutions without moving sensitive patient data to a central location. The FeTS (Federated Tumor Segmentation) initiative, involving 71 institutions globally, serves as a successful model for this collaborative approach.

3. Continuous Biomarkers and Statistical Power

A significant conceptual shift proposed is the move from categorical labels (e.g., "responder" vs. "non-responder") to continuous AI-predicted survival scores. By analyzing the "delta"—the difference between actual survival and AI-predicted baseline survival—researchers can detect treatment effects more efficiently. This approach has the potential to significantly increase the statistical power of clinical trials, even with smaller patient cohorts.

Data Resources and Benchmarking

The roadmap identifies several critical public repositories and specialized benchmarks for model development:

  • Public Repositories: TCIA (The Cancer Imaging Archive), TCGA (The Cancer Genome Atlas), and the UPENN-GBM cohort.
  • Specialized Benchmarks: BraTS for segmentation and molecular characterization, and ACRIN-DSC-MR-Brain for post-treatment progression.
  • Validation Standards: Adherence to reporting guidelines like TRIPOD-AI and PROBAST-AI is essential for building clinician trust.

Key Insights and Future Directions

The AI-RANO group emphasizes that as the WHO classification of CNS tumors evolves toward molecular genetics (e.g., IDH mutation status), AI models must be continuously updated to reflect these new "ground truths." Transparency through open-source code and independent validation on unseen cohorts is non-negotiable for clinical deployment.

While hurdles such as data scarcity for rare tumor types and the need for in-house computational expertise remain, the future lies in Uncertainty Estimation. Teaching models to identify cases that fall outside their training experience—essentially allowing the AI to say "I don't know"—will be crucial for maintaining safety in high-stakes clinical environments.

Summary updated

BibTeX
@inproceedings{bakas2024artificial,
  author = {Spyridon Bakas and Philipp Kickingereder and Norbert Galldiks and Thomas C. Booth and Hugo J.W.L. Aerts and Wenya Linda Bi and Benedikt Wiestler and Pallavi Tiwari and Sarthak Pati and Ujjwal Baid and Evan Calabrese and Philipp Lohmann and Martha Nowosielski and Rajan Jain and Rivka R. Colen and Marwa Ismail and Ghulam Rasool and Janine Lupo and Hamed Akbari and J. C. Tonn and David R. Macdonald and Michael A. Vogelbaum and Susan M. Chang and Christos Davatzikos and Javier Villanueva-Meyer and Raymond Y. Huang},
  title = {{Artificial Intelligence for Response Assessment in Neuro Oncology (AI-RANO), part 2: recommendations for standardisation, validation, and good clinical practice}},
  booktitle = {The Lancet Oncology},
  year = {2024},
  doi = {10.1016/s1470-2045(24)00315-2},
}
Figure