Predictive gains, causal losses: why interventional competence cannot be learned from data alone

Marcus ThomasarXiv 2026 · arXiv

TL;DR

This paper provides a formal "no-go" theorem for causal machine learning: any predictive improvement over a causally correct baseline on observational data requires the model to rely on non-causal features. This reliance creates a mathematical lower bound on error when the model is actually used for interventions, proving that simply "scaling up" observational training can actively damage a model's causal competence.

What Problem Does This Paper Address?

We increasingly use machine learning models—from medical diagnostic tools to Large Language Models (LLMs)—to guide actions, plans, and experiments. These use cases are fundamentally interventional: we are changing the world and asking the model to predict the result.

However, most models are trained on observational data (predicting $Y$ given $X$). The core challenge is that a feature might be great for predicting an outcome in a stable environment but completely irrelevant (or misleading) once you intervene. For example, a model might use "yellow fingers" to predict "lung cancer." This works observationally, but if you intervene by washing someone's hands, the model’s prediction shouldn't change.

The author tackles the question: Does improving observational accuracy necessarily lead to better causal understanding? The paper argues that, mathematically, the answer is often "no."

Methods

The paper is primarily a theoretical work rooted in Structural Causal Models (SCMs) and differential geometry. Key methodological contributions include:

  • The Locality-Autonomy Principle (LAP): A formalization of the idea that a mechanism's output should only depend on its direct causal parents (locality) and that changing one mechanism shouldn't break others (autonomy).
  • Lie Derivative Witnesses: The author uses Lie derivatives to measure how a model's output changes along an "intervention flow." This allows for a geometric test of whether a model respects causal boundaries.
  • Formal Proofs: The paper derives several theorems (notably Lemma 4 and Theorem 6) that quantify the trade-off between observational gain and interventional error.
  • Interventional Alignment: A new framework for checking if a model’s internal state consistently represents the same intervention, regardless of the specific physical process used to realize it.

Datasets

As a theoretical paper, it does not utilize standard large-scale training datasets. Instead, it:

  • Proposes a Synthetic Benchmark: In Section D, the author outlines a controlled arithmetic task for LLMs. It involves two computations ($S = a + b$ and $R = cx + d$). The goal is to see if intervening on the "affine rule" ($R$) erroneously changes the "summation" ($S$), which would witness a failure of computational autonomy.
  • Mathematical Simulations: The proofs use idealized distributions (e.g., unit variance centered variables) to demonstrate how gradient descent behaves under specific causal structures.

Key Insights and Ideas

  • The "No-Go" Result (Lemma 4): If a model achieves a predictive advantage over the "best" causal model (one that only uses direct causes), it must have a non-zero dependence on non-causal inputs. This dependence creates a "causal loss" that is bounded away from zero.
  • Gradients Can Be Causal Saboteurs: The paper shows that standard gradient updates can "damage" a previously correct causal mechanism. If two mechanisms share parameters, optimizing for a global observational loss can force a causally correct sub-component to become "spuriously sensitive" to irrelevant inputs.
  • The Hypothesis Class Trap: Even if you give a model interventional data, it cannot achieve competence if the "correct" causal hypothesis isn't in its admitted class. The limit of learning isn't just data; it's the "causal alternatives" the system is capable of considering.
  • The Autonomy Conjecture: The author conjectures that for a system to truly maintain causal competence in a changing world, it must be an autonomous participant—it must be able to propose its own causal hypotheses, test them, and revise them when they fail.

Key Results

  • Quantitative Error Bounds: Lemma 4 proves that for a predictive gain $G > 0$, the root-mean-square intervention-response error is at least $G / (\sqrt{2}g)$, where $g$ is the strength of the non-causal signal.
  • Gradient Misalignment: Theorem 6 identifies specific conditions where the directional derivative of observational loss is negative (good for training) while the derivative of interventional error is positive (bad for causality). This means the model gets "smarter" at predicting but "dumber" at acting simultaneously.
  • Access-Mode Independence: Proposition 9 shows that the "Model-Class-Adequacy Gap" is independent of whether the learner has access to observational or interventional data; if the hypothesis class is flawed, the error persists regardless of the data source.

Limitations and Future Work

  • Exact vs. Practical Competence: Much of the paper focuses on "exact" structural competence. The author acknowledges that "practical adequacy" (staying within a certain error tolerance $\delta$) is a different, though related, hurdle.
  • The "Autonomous" Requirement: The final conjecture—that autonomy is necessary for causal competence—is a philosophical and technical provocation that remains to be empirically proven in complex AI systems.
  • Implementation Complexity: While the paper proposes a test for LLMs, identifying "intervention directions" in the high-dimensional activation space of a transformer is notoriously difficult (the "attribution problem").
  • Future Direction: The author suggests moving toward "interventional criticism," where models are evaluated not just on their outputs, but on whether their internal computations respect the "Locality-Autonomy Principle."

Summary updated

Abstract

Retaining predictive gains can require persistent errors in a model's response to interventions. For deterministic predictors with finite second moments under squared-error loss, any observational advantage over the best predictor using only an outcome's direct causes requires dependence on other inputs, regardless of the learning rule. When admissible interventions vary inputs while preserving the worldly outcome, this dependence yields a positive lower bound on root-mean-square intervention-response error. We identify conditions under which population gradient updates introduce this dependence from a parent-only baseline or damage a previously correct mechanism through shared parameters while improving other predictions. These dependencies can also make model behavior vary with how the same intervention is realized. We characterize interventional alignment through a common output mapping across internal states and physical processes realizing the same intervention target and relevant context. Correction is limited by the causal hypotheses currently admitted within inquiry. The relevant hypothesis class comprises these causal alternatives, rather than the function class the computational system could in principle represent. For predictions restricted to these alternatives, interventional mean error cannot fall below the best approximation they permit, whether evidence is observational or interventional. Reducing error below a positive limit requires admitting better hypotheses, which may entail revising the causal variables, mechanism structure, or intervention interpretation. Interventional criticism constrains replacements without determining them. We conjecture that reliably acquiring and maintaining interventional competence when causal commitments fail requires autonomous participation in proposing alternatives, testing their consequences, and revising them in response to error.
BibTeX
@misc{thomas2026predictive,
  author = {Marcus Thomas},
  title = {{Predictive gains, causal losses: why interventional competence cannot be learned from data alone}},
  year = {2026},
  eprint = {2611.03861},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03861},
}
Figure