Invariant and Transportable Representations for Anti-Causal Domain Shifts

Yibo Jiang, Victor Veitch

TL;DR

Standard causal domain generalization methods often fail in "anti-causal" settings where the label actually causes the observed features (such as image classification). This paper introduces ACTIR, a framework that leverages the shared causal structure of different domains to disentangle invariant features from domain-varying ones. This allows models to maintain robust performance across domain shifts while enabling rapid adaptation to new environments using only a handful of labeled examples.

What Problem Does This Paper Address?

Domain shift—where a machine learning model performs poorly because the deployment (test) distribution differs from the training distribution—is a pervasive issue in real-world applications.

To tackle this, many researchers turn to causality. A common assumption is that causal relationships remain invariant across domains, leading to methods that try to predict a label $Y$ using only its causal parents. However, many real-world problems are anti-causal: the label $Y$ is actually the cause of the covariates $X$. For example, in image classification, the object class ("dog") determines the pixels in the image ($X$), not the other way around. In these anti-causal settings, $Y$ has no causal parents, rendering traditional causal invariance methods useless.

This paper addresses how to learn representations under anti-causal domain shifts that achieve two goals simultaneously:

  1. Domain Invariance: Learning a fixed predictor that performs reliably across all domains.
  2. Transportability (Rapid Adaptation): Learning a data representation that can be quickly tuned to a novel target domain using very few examples.

Methods

The authors formalize the anti-causal domain shift by introducing two latent (unobserved) variables: $Z$ (the factors of variation whose relationship with $Y$ changes across domains) and $U$ (an unobserved confounder affecting both $Z$ and $Y$).

Figure 1: Causal model for the data generating process. We decompose the observed covariate X into latent parts defined by their causal relationships with Z. Solid circles denote observed variables, while shaded circles denote hidden variables.

As shown in Figure 1, they decompose the features $X$ into two parts:

  • $X_z^\perp$: The part of $X$ not causally affected by $Z$ (the invariant features).
  • $X_z$: The part of $X$ affected by $Z$ (the domain-varying features).

Since $Z$ and $U$ are hidden, the authors derive an observable signature based on a key theorem: if an invariant predictor $g(X)$ depends only on $X_z^\perp$ and a domain-varying representation $h(X)$ depends only on $X_z$, then $g(X)$ and $h(X)$ must be conditionally independent given the label $Y$ ($g(X) \perp\perp h(X) \mid Y$).

To implement this, they introduce ACTIR (Anti-Causal Transportable and Invariant Representation), which optimizes a shared neural network representation $\Phi(X)$ split into invariant and adaptive components. They enforce the conditional independence signature using a novel, computationally efficient causal regularizer ($C_{cond}$). The final objective uses a bi-level optimization strategy with a gradient penalty trick (similar to Invariant Risk Minimization) to ensure the domain-specific predictors are optimal within their respective domains.

Datasets

The authors evaluated ACTIR against several baselines (ERM, IRM, and MAML) using both synthetic and real-world benchmarks:

  • Synthetic Dataset: Data generated directly from anti-causal structural equations.
  • Color MNIST: A variant of MNIST where digit color is strongly but spuriously correlated with the label in a domain-dependent way.
  • Camelyon17: A real-world medical imaging dataset for tumor detection where different hospitals act as distinct domains.
  • PACS: A domain generalization benchmark containing four stylistic domains: Art painting, Cartoon, Photo, and Sketch.
  • VLCS: A photographic dataset compiling images from four distinct computer vision datasets (Caltech, LabelMe, SUN, VOC).

Key Insights and Ideas

  • The Anti-Causal Pivot: Instead of discarding all features affected by the label, the framework isolates the specific components of the features that are influenced by unstable, domain-varying latent factors ($Z$).
  • Disentanglement without Supervision: Remarkably, ACTIR successfully separates invariant features from domain-variant features without ever directly observing or being told what the latent factors (like color or image style) actually are.
  • Isolate, Don't Destroy: Traditional robust methods often completely discard domain-variant features to achieve invariance. ACTIR demonstrates that keeping these features isolated in a separate "compartment" ($h(X)$) allows the model to leverage them for lightning-fast adaptation when a few local samples become available.

Key Results

  • Synthetic & Color MNIST: ACTIR achieved high invariant test accuracy (~74.8% and 70.3% respectively) where standard Empirical Risk Minimization (ERM) failed catastrophically (~10% and 28.2%). Crucially, when given just 10 fine-tuning examples from the test domain, ACTIR's accuracy jumped to 89.28% and 85.25%, significantly outperforming IRM and MAML.
  • Camelyon17: ACTIR achieved the highest invariant test accuracy (77.73%) and demonstrated a much steeper performance gain during few-shot adaptation compared to all baselines.
  • PACS & VLCS: ACTIR maintained highly competitive invariant performance (e.g., 94.17% on PACS Photo) and consistently showed superior adaptation capabilities across most target domains.

Limitations and Future Work

  • Strict Structural Assumptions: ACTIR relies heavily on its specific causal graph. If the assumed conditional independence fails—for instance, if the invariant features directly influence the variant features—the method's performance degrades (as demonstrated by the authors' counterexample in Appendix B).
  • Requirement for Domain Labels: The practical algorithm requires data to be explicitly grouped into distinct training domains to calculate the gradient penalties.
  • Optimization Complexity: The bi-level optimization problem is challenging to solve perfectly; the authors suggest that refining the practical training procedure and finding formal theoretical guarantees for domain diversity are important avenues for future work.

Summary updated

Abstract

Real-world classification problems must contend with domain shift, the (potential) mismatch between the domain where a model is deployed and the domain(s) where the training data was gathered. Methods to handle such problems must specify what structure is common between the domains and what varies. A natural assumption is that causal (structural) relationships are invariant in all domains. Then, it is tempting to learn a predictor for label $Y$ that depends only on its causal parents. However, many real-world problems are "anti-causal" in the sense that $Y$ is a cause of the covariates $X$ -- in this case, $Y$ has no causal parents and the naive causal invariance is useless. In this paper, we study representation learning under a particular notion of domain shift that both respects causal invariance and that naturally handles the "anti-causal" structure. We show how to leverage the shared causal structure of the domains to learn a representation that both admits an invariant predictor and that also allows fast adaptation in new domains. The key is to translate causal assumptions into learning principles that disentangle "invariant" and "non-stable" features. Experiments on both synthetic and real-world data demonstrate the effectiveness of the proposed learning algorithm. Code is available at https://github.com/ybjiaang/ACTIR.
BibTeX
@inproceedings{jiang2022invariant,
  author = {Yibo Jiang and Victor Veitch},
  title = {{Invariant and Transportable Representations for Anti-Causal Domain Shifts}},
  booktitle = {SCIS @ ICML},
  year = {2022},
  url = {https://openreview.net/forum?id=w1FmeUdwxEg},
}
Figure