Abstract
Current causal inference models often rely on "propositional" representations, treating data as flat tables of independent feature vectors. This paper argues that such representations are insufficient for capturing the modularity of real-world systems. By incorporating more expressive representations—including objects, relations, time, space, and hierarchy—researchers can improve the accuracy of causal reasoning, enable automated structure learning, and mitigate biases inherent in simpler models.
Introduction: The "Flat" Representation Problem
While machine learning excels at identifying statistical associations, causal inference seeks to understand the effects of interventions—the "what if" scenarios of active system changes. However, standard causal models, such as the Potential Outcomes Framework or traditional Structural Causal Models, are often "hobbled" by a lack of expressiveness.
Most current models assume data instances are independent and identically distributed (i.i.d.) and represent them as flat feature vectors. This approach ignores the fundamental structure of the physical world, where entities are organized into objects with specific relationships, existing across space and time within complex hierarchies. By failing to capture this modularity, simple models often lead to incorrect causal conclusions and struggle to learn structure from observational data.
The Five Pillars of Expressive Causal Models
To move beyond these limitations, causal modeling must integrate five key representational innovations:
- Objects: Recognizing multiple types of entities with distinct attributes (e.g., distinguishing between birds and blocks).
- Relations: Modeling discrete links between objects, such as one block "supporting" another.
- Time: Accounting for sequences of events and their durations.
- Space: Incorporating spatial embeddings and relative positions.
- Hierarchy and Composition: Understanding systems as assemblies of subsystems (e.g., a platform composed of multiple blocks).
These richer representations allow for "relational d-separation" and object-oriented Markov Decision Processes (MDPs), which have already demonstrated success in broader AI research.
Case Study: Causal Reasoning in Angry Birds
The video game Angry Birds serves as a primary conceptual benchmark for why "flat" models fail. A human player can predict how a structure will collapse because they intuitively understand objects and physics. In contrast, a standard causal model treating the screen as a grid of pixels or a flat list of coordinates would struggle to represent the dynamic, changing relationships between blocks.
A modular causal model treats the game environment as a set of independent components. If a bird hits a specific block, the internal mechanisms of other distant blocks remain unchanged. This modularity is essential for accurate causal discovery; if changing one variable requires rewriting the entire system's logic, the model has failed to capture the underlying causal structure.
Empirical Benefits of Expressiveness
Synthesizing recent research, several key advantages of expressive models emerge:
- Enhanced Structure Learning: Explicitly representing object types and relations allows models to infer the direction of causal dependence ($X \to Y$ vs. $Y \to X$) in scenarios where flat representations find the direction unidentifiable.
- Bias Reduction: Studies have shown that failing to account for object-relational structures leads to "spurious" causal judgments—identifying dependencies that do not actually exist.
- Improved Estimation: Standard causal effect estimators can be strongly biased if they ignore existing relational dependencies in the data, a flaw that expressive models are designed to correct.
Challenges and Future Directions
Increasing model expressiveness is not without cost. The search space for a model incorporating objects, space, and time is significantly larger than that of a simple variable-based model, making learning and inference computationally intensive.
Future research must focus on:
- Efficient Search Algorithms: Developing methods to navigate high-dimensional, expressive model spaces.
- Identifiability: Determining when a causal effect can be uniquely calculated from complex, expressive models.
- Bridging the Gap: Connecting high-level human reasoning (which is naturally object-oriented) with the low-level statistical tools currently used in causal discovery.
Summary updated