Introduction
As machine learning (ML) systems are increasingly integrated into critical sectors such as healthcare and finance, the "right to explanation" has become a cornerstone of global AI policy. However, current transparency frameworks often overlook a fundamental technical distinction: the difference between interpolation and extrapolation.
Most ML models are designed to identify patterns within the "convex hull"—the geometric boundary—of their training data. When a model receives a query that falls outside this boundary, it is no longer making an informed prediction based on experience; it is essentially guessing in the dark. This paper argues that the failure of models to report when they are extrapolating is a "silent failure" mode that undermines human oversight and should be addressed through regulatory mandates.
The Problem: The "Cluelessness" Gap
Currently, ML models provide predictions for unfamiliar data with the same level of apparent confidence as they do for familiar data. This lack of transparency can lead to unreliable medical diagnoses or unfair financial rejections without any red flags being raised for human intervention.
While some researchers argue that extrapolation is rare or inevitable in high-dimensional spaces, this paper demonstrates that in common "mid-range" datasets (10–50 features), extrapolation is both frequent and significant. Without a mechanism to flag these instances, stakeholders—such as physicians or loan officers—cannot accurately assess the risk of a model's output.
Methodology: Geometric Verification
The authors propose a conceptual and technical framework grounded in geometric principles to bridge this gap:
- Convex Hull Verification: The researchers suggest using mathematical algorithms to determine if a new data point lies within the training set's convex hull. They liken the training data to a "dome"; if a point falls outside this dome, the model is extrapolating.
- Directional Reporting: Beyond a simple binary flag, the authors propose that models should report the direction and extent of extrapolation. This involves identifying which specific features (e.g., age, education level, or medical history) are pushing the sample into unfamiliar territory.
- Computational Efficiency: A critical finding of the paper is that checking for extrapolation adds a negligible computational burden compared to the training and inference requirements of complex modern models, making it a practical addition to real-world systems.
Empirical Evidence
To prove that extrapolation is a pervasive phenomenon rather than a marginal edge case, the authors evaluated two major datasets:
- Adult Income Dataset: In this standard benchmark for social ML applications, approximately 50% of the testing samples required some level of extrapolation. In several instances, models extrapolated across multiple dimensions like age and race simultaneously, leading to potentially unsound decisions.
- Veterans Affairs (VA) Healthcare System Dataset: Analyzing over one million patient records, the authors found that in a 5-fold cross-validation, roughly 15% of patient records required extrapolation. From a clinical perspective, many of these cases were deemed too extensive to be considered reliable without expert human review.
Policy Recommendations
The authors argue that the solution is not to prohibit extrapolation—which is often necessary for the flexibility of AI—but to mandate transparency. They provide concrete suggestions for regulatory integration, specifically recommending that a clause be added to Article 13 of the EU AI Act. This clause would require high-risk AI systems to report when they are extrapolating and specify which attributes are involved.
Much like government spending reports or clinical trial disclosures, making extrapolation data public allows stakeholders to identify and scrutinize potential errors or biases. This transparency ensures that human experts remain the final arbiters in high-stakes decisions where the model is effectively "clueless."
Conclusion and Future Work
Reporting extrapolation is a fundamental requirement for AI transparency. While this is only one piece of the broader explainability puzzle—knowing a model is clueless does not explain why it reached a specific conclusion—it provides a vital risk signal.
Future research should focus on defining "acceptable" thresholds for extrapolation and developing intuitive interfaces to present extrapolation attributes to non-technical stakeholders. By formalizing the reporting of extrapolation, we can move toward a more honest and accountable implementation of artificial intelligence.
Summary updated