Continual Causal Abstractions

Matej Zecevic, Moritz Willig, Florian Peter Busch, Jonas Seng

Abstract

This short paper discusses continually updated causal abstractions as a potential direction of future research. The key idea is to revise the existing level of causal abstraction to a different level of detail that is both consistent with the history of observed data and more effective in solving a given task.

Introduction

In both AI and cognitive science, modeling the world at the level of fundamental particles is rarely feasible or useful. Instead, agents rely on abstractions—simplified models that group complex variables into functional summaries, such as using "temperature" rather than tracking individual molecular velocities.

However, a fixed abstraction often fails when an environment changes or when "outlier" data emerges that the coarse model cannot explain. Traditional machine learning approaches frequently suffer from "catastrophic forgetting" or an inability to adapt causal logic to novel evidence. This paper addresses the intersection of causal abstraction (relating different levels of detail) and continual learning (updating knowledge over time). The central question is: How can an agent revise its causal model to a new level of detail that remains consistent with past data while successfully addressing new challenges?

Methods and the CCA Vision

The framework builds upon the mathematical foundation of Structural Causal Models (SCMs) and the theory of $\tau$-abstractions. A $\tau$-abstraction is a formal mapping ensuring that a high-level (coarse) model and a low-level (detailed) model are "consistent." Specifically, interventions on the detailed model should yield results predictable by the coarse model.

The authors propose a four-step vision for implementing Continual Causal Abstractions (CCA):

  1. Initial Abstraction: Establish a parameterized SCM based on initial data.
  2. Rule Extraction: Derive decision rules (e.g., "If X happens, then Y follows").
  3. Inconsistency Detection: Identify model failures when new data (a "counterexample") contradicts existing rules.
  4. Abstraction Learning: Optimize a new mapping ($\tau$) to find a more granular level of abstraction that explains both historical data and the new counterexample.
The CCA framework illustrated through a dietitian example where a model is revised to account for specific cholesterol types (HDL/LDL) when faced with a counterexample.

Illustrative Example: The Dietitian's Dilemma

To demonstrate the theory, the authors use a synthetic "Dietitian Example." Initially, a model relates a "Balanced Diet" (DP) to "Heart Disease" (HD) via "Total Cholesterol" (TC). This coarse model works until a "New Example"—such as a Sumo practitioner—is introduced. This individual may have high total cholesterol but low heart disease risk, creating an inconsistency. To resolve this, the CCA algorithm revises the abstraction to include finer details: High-Density Lipoprotein (HL) and Low-Density Lipoprotein (LL). This more detailed model maintains consistency with previous data while correctly predicting the health outcomes for the new case.

Key Insights

  • Consistency and Commutativity: A revised model must not only fit new data but also "commute" with the previous model. The new, detailed logic must explain why the old, coarse logic was effective for prior cases.
  • Task-Driven Abstraction: Abstraction is not merely for memory efficiency; it is a tool for finding "effective decision rules." The appropriate level of detail is dictated by the specific "Scientific Question" the agent aims to answer.
  • Revision as a "Different Lens": When a model fails, the solution is often not just more data, but a shift in perspective. Moving between macroscopic and microscopic views is a fundamental component of intelligent learning.

Limitations and Future Research

While providing a formal roadmap, several challenges remain:

  • The Search Problem: Efficiently searching for the optimal mapping ($\tau$) is difficult, as the number of possible abstractions in complex systems is astronomical.
  • Feature Discovery: The current framework assumes "finer" features (like HDL/LDL) are present in the data but ignored. Real-world agents may need to actively discover or measure entirely new variables.
  • Automation: General, scalable systems for verifying if one SCM is a valid abstraction of another do not yet exist, marking a significant frontier for future research.

Summary updated

Abstract

This short paper discusses continually updated causal abstractions as a potential direction of future research. The key idea is to revise the existing level of causal abstraction to a different level of detail that is both consistent with the history of observed data and more effective in solving a given task.
BibTeX
@inproceedings{zecevic2023continual,
  author = {Matej Zecevic and Moritz Willig and Florian Peter Busch and Jonas Seng},
  title = {{Continual Causal Abstractions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=QbTWkWu1Ky},
}
Figure