Introduction
In recent years, artificial intelligence (AI) has achieved superhuman performance in complex scientific domains, from weather forecasting to protein folding. However, for the scientific community, a correct prediction is only a partial victory. The traditional scientific method is rooted in understanding the underlying principles—the "why" behind a phenomenon.
Currently, many high-performing AI models function as "black boxes." While their predictive power is undeniable, the specific patterns or data relationships they exploit often remain opaque. This creates a significant "knowledge gap": AI may have uncovered novel principles of nature that remain inaccessible to human understanding. Without robust interpretability, AI risks becoming a dead-end for scientific insight rather than a catalyst for discovery.
The XAI for Science Framework
To bridge this gap, this paper proposes a framework called Explainable AI for Science. This approach moves beyond using AI solely for prediction, repositioning it as a tool for active scientific discovery. By integrating interpretability methods into the classical inductive reasoning process, researchers can extract a "machine view" of a problem.
The framework relies on three core components:
- The Machine View: Utilizing interpretability tools to identify which data features the AI prioritizes when making a decision.
- The ARU Pillars: To ensure scientific utility, the "machine view" must satisfy three criteria: accuracy (faithfully representing the model), reproducibility (consistency across trials), and understandability (relatability to domain knowledge).
- Interpretability-Guided Explanations (IGEs): This is the synthesis stage where human scientists (potentially aided by Generative AI) compare the "machine view" with the "human view" (existing knowledge).
The paper distinguishes between post-hoc interpretability (explaining a pre-trained model using tools like SHAP or GradCAM) and ante-hoc interpretability (employing inherently transparent models like decision trees).
Case Studies: Validating the Workflow
The proposed workflow was tested using two distinct datasets, with Transformer neural networks serving as the predictive engines:
- Medical Diagnostics (ECG Time-Series): In classifying heart pathologies like Right Bundle Branch Block (RBBB), the AI (via SHAP) identified the delayed and broadened R'-wave in the V1 lead as the primary diagnostic feature. This convergent view aligns with established medical principles, validating the AI's trustworthiness.
- Climate Science (Spatio-temporal Weather Data): For forecasting extreme heatwaves in Indochina, the AI prioritized rising temperatures (T2M) and high-pressure anomalies (SLP) while correctly identifying low precipitation as a minor factor. This allows scientists to pinpoint the specific atmospheric "windows" the AI monitors.
Discussion and Future Directions
A key insight of this framework is that divergence between human and machine views is not a failure, but a catalyst for inquiry. When an AI identifies a feature that contradicts or extends human knowledge, it signals a potential new scientific discovery.
Furthermore, the paper explores the role of Generative AI (GenAI) as a "scientist's assistant." Large Language Models (LLMs) can help bridge the gap by translating raw interpretability data into human-readable hypotheses, acting as a bridge between the "machine view" and the "human view."
However, challenges remain. AI is prone to finding correlations rather than causality, leading to potential "scientific hallucinations." Additionally, while we can see what data a model uses, understanding how it mathematically relates those features is still difficult. Future work may involve symbolic regression to transform these data patterns into explicit mathematical laws, fully integrating AI into the scientific method.
Summary updated