Group invariance principles for causal generative models

Michel Besserve, Naji Shajarisales, Bernhard Schölkopf, Dominik Janzing

Introduction

The Independence of Cause and Mechanism (ICM) principle is a cornerstone of causal inference, suggesting that in a causal relationship $X \to Y$, the process generating the cause $P(X)$ and the mechanism $P(Y|X)$ are independent modules of nature. This independence implies a lack of "fine-tuning" or specific alignment between the two. If a mechanism appears suspiciously adapted to its input—much like a key fitting a complex lock—it suggests a non-generic relationship, often indicating an incorrect causal direction or model misspecification. This paper introduces a unified mathematical framework based on group theory to formalize and test this notion of genericity across various data types and models.

Theoretical Framework

The proposed framework formalizes "independence" through three pillars: a generative model $m$, a group of transformations $G$ (such as rotations, permutations, or translations), and a contrast function $C$ that measures global properties of the output.

The proposed framework (a) applies a random transformation $g$ to input $x$ to test the genericity of mechanism $m$, aligning with the causal structure shown in (b).

The core of this approach is the Genericity Equation. For a truly causal model, the contrast of the observed output $C(m(x))$ should be "typical" relative to the distribution of contrasts obtained by applying random transformations from the group to the input: $C(m(g(x)))$ where $g \in G$. If the observed contrast is an outlier, the model is considered atypical, signaling that the mechanism and cause are improperly aligned.

Conceptual Intuition: Atypicality as a Diagnostic

The concept of atypicality is analogous to human visual perception and optical illusions. For instance, the Kanizsa Triangle relies on a very specific, "atypical" alignment of shapes to create the illusion of a foreground object. A random rotation of these shapes would immediately destroy the effect, revealing the alignment as non-generic.

Visualizing atypicality: (a) The Kanizsa Triangle illusion relies on specific alignments. (b-f) Geometric examples where "suspicious" alignments indicate non-generic relationships.

In machine learning, this principle acts as a red flag. If a model's output property is highly sensitive to input transformations, it suggests the model has overfit the relationship between cause and effect. The authors demonstrate that several existing causal methods, such as the "Trace Method" for linear models, are specific instances of this broader group-theoretic framework.

Applications and Empirical Results

The framework was validated using synthetic simulations in Non-negative Matrix Factorization (NMF) and Gaussian Mixture Model (GMM) clustering:

  • NMF Diagnostics: The "generic ratio" (observed contrast divided by expected generic contrast) effectively identifies when an algorithm fails to recover true components or when the number of components is overestimated.
  • Clustering Validation: In GMMs, the framework detects "pathological" clustering, such as when a single natural cluster is incorrectly split into two, resulting in centroids and covariances that exhibit atypical dependencies.
  • GANs as Causal Learners: The authors argue that Generative Adversarial Networks (GANs) implicitly utilize this principle. The discriminator functions as a learned contrast function, forcing the generator to produce data that is generic relative to the real data distribution.

Conclusion and Future Directions

By unifying fragmented causal methods under the language of group theory, this research provides a powerful diagnostic tool for assessing the causal plausibility of generative models. While the framework currently requires manual selection of the appropriate group and contrast function for a given domain, it opens new avenues for research. Future work may explore using group invariance as a regularization technique to ensure neural networks learn robust, causal representations rather than mere statistical correlations.

Summary updated

BibTeX
@inproceedings{besserve2018group,
  author = {Michel Besserve and Naji Shajarisales and Bernhard Schölkopf and Dominik Janzing},
  title = {{Group invariance principles for causal generative models}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2018},
  url = {https://openreview.net/forum?id=r-jIb6lwNfS},
}
Figure