Introduction
While uncertainty is a cornerstone of machine learning, its specific intersection with causality has remained fragmented. Traditional machine learning typically distinguishes between aleatoric uncertainty (inherent randomness in data) and epistemic uncertainty (lack of model knowledge). However, causality introduces unique challenges that this binary view fails to capture. For instance, how do we categorize the doubt regarding whether a relationship is a genuine cause-and-effect link or a mere correlation? Without a unified taxonomy and quantification framework, building AI systems—especially Large Language Models (LLMs)—that can reliably reason about "what-if" scenarios in high-stakes domains like medicine or finance remains an elusive goal.
This paper provides the first systematic survey of uncertainty within the field of causality, introducing a novel trichotomy to categorize causal doubts and evaluating the current limitations of LLMs in this space.
A New Taxonomy: The Causal Trichotomy
The authors propose extending the traditional ML dichotomy by adding a third, crucial category: Ontological Uncertainty. This refers to existential doubts about the causal link itself. For example, is the observed relationship between "ice cream sales" and "shark attacks" a real cause, or is there a hidden confounder like "hot weather"?
As shown in the taxonomy above, these categories are further broken down:
- Aleatoric Uncertainty: Derived from inherent randomness or measurement noise.
- Epistemic Uncertainty: Stemming from incomplete knowledge about causal structures or parameters.
- Ontological Uncertainty: Related to the very existence of a causal link, contextual validity, or probabilistic nature.
Quantification and LLM Evaluation
The paper reviews two complementary methods for measuring these uncertainties: Direct Estimation (using entropy or verbalized expressions like "possibly") and Causal Strength Estimation (measuring how much a cause increases the likelihood of an effect using Probability Raising Theory).
A significant portion of the survey evaluates how current LLMs handle these concepts using benchmarks like e-CARE (explainable reasoning), XCOPA (commonsense), and Cladder (correlation vs. causation). The authors identify a "hallucination over causal uncertainty," where models act as "causal parrots"—confidently asserting causal links based on training patterns rather than authentic reasoning.
Key findings from the evaluation include:
- Pattern Memorization: LLMs often succeed at causal tasks by memorizing data patterns. When faced with ambiguous evidence, they frequently self-contradict or hallucinate spurious links.
- Consistency Issues: Research indicates that smaller models (≤ 7B parameters) often perform near random baselines when tasked with maintaining consistency in cause-effect ranking.
- Verbalized Confidence: Interestingly, LLMs' use of "modal words" (e.g., "maybe") is often better calibrated than their raw probability scores, suggesting a promising path for human-AI interaction.
Towards Optimal Causal AI
To move beyond "causal parrots," the authors outline three pillars for future robust, uncertainty-aware causal AI:
- Consistency: Decisions must be stable and reproducible across different contexts.
- Versatility: Models should handle all levels of Pearl’s "Ladder of Causation," from associational to counterfactual reasoning.
- Accurate Estimation: The ability to identify, anticipate, and quantify all three types of causal uncertainty (Aleatoric, Epistemic, and Ontological).
Conclusion and Future Work
While the proposed trichotomy provides a much-needed roadmap, the authors acknowledge that boundaries between categories can blur in complex scenarios. Future research must focus on creating numerical benchmarks for causal uncertainty and developing models capable of "counterfactual quantification"—predicting not just what might happen, but exactly how uncertain that prediction is.
Summary updated