Clinical predictions using artificial intelligence in neurosurgery and its subspecialties: a systematic review and meta-analysis

Prajwal Ghimire, Aswin Chari, Jonathan Shapey, Keyoumars AshkanarXiv 2026 · DOI · arXiv

TL;DR

This massive meta-analysis of over 1,000 studies reveals that while AI models in neurosurgery report impressive performance (average AUC of 0.86), the field faces a severe "translation gap." Most models are developed in isolation using retrospective data, suffer from high risk of bias, and lack the external validation and calibration necessary for safe clinical deployment.

What Problem Does This Paper Address?

The integration of Artificial Intelligence (AI) into neurosurgery is moving at breakneck speed, with models promising to predict everything from tumor subtypes to post-surgical complications. However, clinical enthusiasm has largely outpaced methodological rigor.

The authors address a critical uncertainty: Do these models actually work well enough for the operating room, or are we looking at a bubble of "optimism-prone" research? By systematically reviewing the literature, they aim to map the current state of AI in neurosurgery, evaluate its performance across different subspecialties, and identify the bottlenecks preventing these tools from reaching actual patients.

Methods

The researchers conducted a systematic review and meta-analysis following PRISMA guidelines, searching PubMed and Ovid for peer-reviewed studies published between 2015 and 2026.

Key methodological steps included:

  • Performance Pooling: They used random-effects modeling to pool metrics like Area Under the Curve (AUC), accuracy, sensitivity, and specificity.
  • Categorization: Studies were classified by neurosurgical subspecialty (e.g., neuro-oncology, spine) and their position in the clinical care pathway (from pre-diagnosis to post-operative care).
  • Quality Assessment: The team used the PROBAST (Prediction model Risk Of Bias Assessment Tool) to evaluate the risk of bias and the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) guidelines to assess reporting quality.
  • Translation Ladder: They assigned each study a "translation stage" (T0 to T5) to see how close the models were to real-world use.

Datasets

As a meta-analysis, this paper does not introduce a new primary dataset. Instead, it synthesizes data from 1,063 original studies.

  • Subspecialty Distribution: Neuro-oncology was the most represented (44%), followed by spine (17.3%) and neuro-vascular (16.9%).
  • Data Types: The underlying studies primarily utilized medical imaging (MRI/CT), clinical electronic health records, and radiomics features.
Figure 1. PRISMA-based search strategy of the systematic review for AI-based clinical prediction tools in neurosurgery. AUC: area under the curve

Key Insights and Ideas

  • The "Easy" vs. "Hard" Tasks: AI performs best at imaging-anchored diagnostic tasks (e.g., identifying a tumor's molecular grade), where the ground truth is currently present. It struggles more with prognostic tasks (e.g., predicting a future complication), which are arguably more valuable for surgical decision-making.
  • The Analysis Bottleneck: A staggering 89% of studies were at high risk of bias, primarily driven by the "analysis" domain. Common failures included poor handling of missing data and a total lack of model calibration (ensuring the predicted probability matches the actual observed frequency).
  • Internal Validation Trap: 58% of models relied solely on internal validation or reported no validation at all. This leads to "optimism bias," where models look great on the data they were built on but fail when exposed to new patient populations.
  • Algorithm Trends: While deep learning is popular, traditional "Regression-based ML" (like Logistic Regression and LASSO) still accounts for a plurality of the models used in the field.
Distribution of algorithms and model families across the 1,063 studies.

Key Results

  • High Aggregate Performance: The overall pooled AUC was 0.86 (95% CI: 0.85–0.86), which is generally considered "excellent" discrimination.
  • Subspecialty Consistency: Performance was remarkably consistent across subspecialties, ranging from 0.84 in spine to 0.87 in neuro-oncology.
  • The Translation Gap: Only 4.9% of studies reached the stage of clinical use or prospective evaluation. Most research (63.9%) stopped at initial model development.
  • Extreme Heterogeneity: The $I^2$ value was 98.6%, indicating that the results between studies vary wildly. This suggests that a single "pooled AUC" doesn't tell the whole story; individual model performance is highly dependent on the specific task and dataset.
Pooled AUC scores across neurosurgical subspecialties show high but variable performance.

Limitations and Future Work

The authors acknowledge that the extreme heterogeneity in the literature makes it difficult to draw definitive conclusions about which specific AI architectures are "best." Furthermore, the widespread lack of reported confidence intervals in many studies forced the authors to impute standard errors, which may introduce minor inaccuracies in the meta-analysis.

The Path Forward: The authors argue that the neurosurgical community must move past the "AUC arms race." Future research should prioritize:

  1. External Validation: Testing models on geographically and temporally distinct datasets.
  2. Calibration Analysis: Reporting how well predicted risks align with actual outcomes.
  3. Clinical Impact Studies: Moving beyond "silent" testing to see if AI actually changes surgical decisions or improves patient recovery.
  4. Adherence to Reporting Standards: Using frameworks like TRIPOD-AI to ensure transparency and reproducibility.
Figure 4. Machine learning prediction studies in neurosurgery by clinical-pathway stage and subspeciality. dx: diagnostic

Summary updated

Abstract

Artificial intelligence models for clinical prediction are proliferating across neurosurgery, but their aggregate discriminative performance, distribution along the care pathway and methodological quality are unclear. We systematically reviewed and meta-analysed these studies in neurosurgery and its subspecialities along with their risk of bias appraisal. We performed a PRISMA search for peer-reviewed original studies using AI to predict a clinical outcome in neurosurgical populations. Metrics were pooled using random effects modelling. Each study was classified as diagnostic, prognostic or forecasting and was assigned an evidence level. Clinical prediction task sub-stratification analysis was performed. Risk-of-bias was assessed using PROBAST. 1063 studies were included. Across 451 studies reporting AUC, pooled discrimination was 0.86, ranging from 0.84 (spine) to 0.87 (neuro-oncology), with extreme heterogeneity. Performance varied by clinical pathway stage, highest for diagnosis, grading and molecular tasks and lowest for post-operative tasks. Only 51 studies were using prospective data with external validation, whereas 619 rested on internal validation alone or no validation. 89 percent studies were at high risk-of-bias, driven by the analysis domain. AI models report high performance across all the neurosurgical subspecialities, concordant across AUC, accuracy, sensitivity and specificity and robust to risk-of-bias stratification. Performance is highest for imaging-anchored diagnostic tasks and lowest at decision-facing post-operative stage. However, the literature is dominated by optimism-prone, internally-validated and poorly calibrated models with extreme heterogeneity. External validation, calibration analysis and reporting guidelines adherence remain prerequisites for clinical adoption.
BibTeX
@misc{ghimire2026clinical,
  author = {Prajwal Ghimire and Aswin Chari and Jonathan Shapey and Keyoumars Ashkan},
  title = {{Clinical predictions using artificial intelligence in neurosurgery and its subspecialties: a systematic review and meta-analysis}},
  year = {2026},
  eprint = {2611.12385},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12385},
}
Figure