BISCUIT: Causal Representation Learning from Binary Interactions

Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M. Asano, Taco Cohen, Efstratios Gavves

Abstract

Identifying the causal variables of an environment and how to intervene on them is of core value in applications such as robotics and embodied AI. While an agent can commonly interact with the environment and may implicitly perturb the behavior of some of these causal variables, often the targets it affects remain unknown. In this paper, we show that causal variables can still be identified for many common setups, e.g., additive Gaussian noise models, if the agent’s interactions with a causal variable can be described by an unknown binary variable. This happens when each causal variable has two different mechanisms, e.g., an observational and an interventional one. Using this identifiability result, we propose BISCUIT, a method for simultaneously learning causal variables and their corresponding binary interaction variables. On three robotic-inspired datasets, BISCUIT accurately identifies causal variables and can even be scaled to complex, realistic environments for embodied AI.

Introduction

In the quest for truly intelligent robots and embodied AI, we want agents to understand the "knobs" of their environment—the underlying causal variables. However, these variables are usually hidden behind high-dimensional observations like pixels. The core challenge in learning these variables is identifiability: many different causal models can produce the exact same set of images, making it mathematically impossible to know if a model has found the "true" causal factors or just a convenient rotation of them.

Existing solutions often require "supervised" interventions (knowing exactly which variable was changed) or rely on very strict assumptions about noise that don't hold up in real-world scenarios. BISCUIT (Binary Interactions for Sequential Causal Unsupervised IdenTification) tackles this by asking: Can we identify causal variables simply by knowing that an agent is interacting with the world, even if we don't know exactly what it's hitting?

The BISCUIT Framework

BISCUIT is built on a Variational Autoencoder (VAE) architecture with a specialized prior designed to capture the dynamics of an environment under agent interaction.

Overview of the BISCUIT framework showing how observations and agent actions are mapped to latent causal variables via binary interaction switches.

Key components of the architecture include:

  • The Regime Variable ($R^t$): An observable signal of the agent's intent or action (e.g., the 2D coordinates of a robot arm).
  • Latent Binary Interactions ($I^t_i$): For every latent variable the model learns, it also learns a binary "interaction" variable. This variable acts as a gate, switching between two mechanisms: an "observational" one (how the variable behaves naturally) and an "interventional" one (how it behaves when touched or changed).
  • BISCUIT-NF: For complex visual scenes, the authors use a two-stage approach. First, an autoencoder compresses images into a latent space; then, a Normalizing Flow (NF) transforms that space into the causal representation.

Causal Structure and Identifiability

The paper provides a theoretical foundation for identifiability. It proves that if interactions follow "distinct patterns" (i.e., the agent doesn't always interact with the exact same set of variables simultaneously), the causal factors become identifiable. This holds even for the "difficult" case of additive Gaussian noise, provided there is enough Dynamics Variability (non-linear changes) or Time Variability (interactions that depend on the previous state).

The causal graph illustrating the relationships between observations, causal variables, binary interactions, and regimes.

Unlike previous methods that need to be told "the robot is touching the microwave," BISCUIT learns to infer which latent variables are being affected by the agent's actions ($R^t$) entirely from the data. If an interaction depends on the environment's state (e.g., a robot only opens a door if it's standing in front of it), that dependency actually makes the causal variables easier to identify.

Experimental Results

The researchers evaluated BISCUIT on three increasingly complex benchmarks: the synthetic Voronoi Benchmark, the 3D robotic simulation CausalWorld, and the high-fidelity iTHOR kitchen environment.

  • Superior Disentanglement: On the Voronoi benchmark, BISCUIT achieved near-perfect identification ($R^2 \approx 0.95+$), while baselines like iVAE and LEAP struggled to separate the variables, often mixing multiple causal factors into a single latent dimension.
  • Real-World Scaling: In the complex iTHOR kitchen environment, which features 18 causal variables (e.g., microwave states, stove heat), BISCUIT-NF achieved an $R^2$ score of 0.96, significantly outperforming the next best baseline (0.63).
  • Actionable Representations: The authors demonstrated "latent interventions." By encoding two different images and swapping specific causal latents (e.g., taking the "stove is on" state from Image A and the "egg is raw" state from Image B), they could generate realistic images of entirely new, unseen scenarios.
  • High Precision: The model identified the correct interaction targets with an average F1 score of 98% on synthetic data.

Conclusion

BISCUIT demonstrates that simple binary "on/off" switches for mechanisms provide enough signal to anchor latent variables to their true physical counterparts. While the current theory assumes binary interactions and diverse interaction patterns, it paves the way for unsupervised causal discovery in complex, embodied AI environments. Future work may extend this to continuous interactions and longer-term temporal dependencies.

Summary updated

Abstract

Identifying the causal variables of an environment and how to intervene on them is of core value in applications such as robotics and embodied AI. While an agent can commonly interact with the environment and may implicitly perturb the behavior of some of these causal variables, often the targets it affects remain unknown. In this paper, we show that causal variables can still be identified for many common setups, e.g., additive Gaussian noise models, if the agent’s interactions with a causal variable can be described by an unknown binary variable. This happens when each causal variable has two different mechanisms, e.g., an observational and an interventional one. Using this identifiability result, we propose BISCUIT, a method for simultaneously learning causal variables and their corresponding binary interaction variables. On three robotic-inspired datasets, BISCUIT accurately identifies causal variables and can even be scaled to complex, realistic environments for embodied AI.
BibTeX
@inproceedings{lippe2023biscuit,
  author = {Phillip Lippe and Sara Magliacane and Sindy Löwe and Yuki M. Asano and Taco Cohen and Efstratios Gavves},
  title = {{BISCUIT: Causal Representation Learning from Binary Interactions}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=FoE1kpX5CB},
}
Figure