Introduction
In traditional causal discovery, researchers typically assume that causal variables are provided as clean, structured data. However, an autonomous agent in a real-world environment must perform causal induction: it must learn high-level representations directly from raw pixels while simultaneously uncovering the causal "rules of the game." This paper introduces a benchmarking suite to evaluate how reinforcement learning (RL) agents discover these structures and compares various world model architectures.
Methodology
The authors evaluate four types of world models within a Model-Based Reinforcement Learning (MBRL) framework:
- Monolithic Models (AE/VAE): Standard autoencoders that compress the entire visual scene into a single latent vector.
- Graph Neural Networks (GNNs): Models like C-SWM that represent the world as a collection of objects and learn pairwise interactions.
- Modular Transition Models (MTM): Architectures that factorize representations for each object and learn directed rules to model how one object's state influences another.
These models are trained using either Maximum Likelihood (NLL) (pixel-level prediction) or Contrastive Learning (state discrimination).
Experimental Framework
The evaluation utilizes two new environment suites designed for parametric control of causal complexity:
- Physics (Weighted-block pushing): Blocks have hidden weights; heavier blocks can push lighter ones. Agents must infer this hierarchy.
- Chemistry: Objects change colors based on a Directed Acyclic Graph (DAG), where changing a "parent" object triggers a chain reaction in its "children."
Datasets and Complexity
The benchmarking suite allows researchers to manipulate the number of variables, the sparsity of the causal graph, and the length of causal chains. This systematic control reveals how different architectures handle increasing complexity.
Key Insights and Results
- The Pairwise Limitation of GNNs: While GNNs excel at simple pairwise interactions, they struggle with long causal chains (e.g., A causes B, which causes C). Modular models handle these structures more effectively due to their flexible directed relationship modeling.
- Scalability: Modular models scale significantly better than monolithic ones. In environments with 5 objects, Modular models achieved nearly double the success rate of VAEs in downstream RL tasks.
- Zero-Shot Transfer: Structured models demonstrate superior transfer capabilities. For instance, a modular model that learns a weight-color relationship can immediately apply that logic to novel colors it has never encountered.
- Evaluation Metrics: A key finding is that common metrics like "Hits at Rank 1" do not always correlate with actual RL performance. High prediction accuracy does not necessarily translate to effective planning.
Conclusion
The study demonstrates that modular, object-factored architectures are critical for effective causal discovery from visual data. Future research should focus on continuous interventions and handling unobserved confounders to bridge the gap toward real-world robotics.
Summary updated