Abstract
The postulate of independence of cause and mechanism (ICM) has recently led to several new causal discovery algorithms. The interpretation of independence and the way it is utilized, however, varies across different methods. This paper introduces a unifying group-theoretic framework for the ICM postulate, providing a mathematical tool to distinguish true causal directions from "suspicious" alignments, extending these principles from simple two-variable problems to complex tasks like unsupervised clustering.
Introduction
In classical statistics, distinguishing between "X causes Y" and "Y causes X" using only observational data is a fundamental challenge. The Independence of Cause and Mechanism (ICM) postulate addresses this by assuming that the process generating the cause ($P(\text{cause})$) and the mechanism mapping it to the effect ($P(\text{effect}|\text{cause})$) are chosen independently by nature and contain no information about each other.
Existing algorithms based on ICM, such as the Trace Method or IGCI, often appear as a collection of disparate "recipes" for specific data types. This work seeks to unify these methods under a single, principled framework that defines "independence" when comparing fundamentally different objects, such as probability distributions and functions.
The Group Invariance Framework
The core of the proposed framework is Group Invariance and the Expected Generic Contrast (EGC). The process involves four key steps:
- Define a Group: Select a group of transformations (e.g., rotations, translations, or permutations) representing "generic" changes to the system.
- Perturb the System: Apply a random transformation from this group to "break" the existing relationship between the cause and the mechanism.
- Measure Contrast: Use a contrast function to evaluate a property of the system, such as the number of edges in a vision task or the trace of a covariance matrix in a linear model.
- The Genericity Equation: In a truly causal system, the contrast of the original data should roughly equal the average contrast of its perturbed versions. If the original data has a significantly lower or higher contrast, it suggests a "suspicious" alignment—meaning the model is likely "atypical" or non-causal.
By selecting different groups and contrast functions, the authors re-derive several famous causal discovery algorithms (Trace Method, IGCI, and LiNGAM) as special cases of this single framework.
Intuition: Suspicious Alignments and Illusions
Visualizing "suspicious alignments" through the Kanizsa triangle and geometric perturbations. Source, p. 2
The authors use the Kanizsa triangle visual illusion to explain causality. In a "typical" scene (Figure 1b), rotating an object does not fundamentally change the number of visible lines. In an "atypical" scene or illusion (Figure 1a), a minor rotation "breaks" the alignment (Figure 1c), suddenly revealing more lines. This "suspicious alignment" serves as a signal that the model is not the true causal one. The framework formalizes this intuition mathematically through a "genericity ratio."
Applications and Results
Validation on Real-World Data
The ICM postulate was validated using the CIFAR-10 and MNIST datasets. The authors calculated a genericity ratio, which for these natural image distributions was consistently close to 1.0 (e.g., 0.999), confirming that the ICM postulate holds true for these real-world datasets.
Causal Clustering
Applying group invariance to detect atypicality in Gaussian Mixture Model clustering. Source, p. 7
The framework also extends to unsupervised learning. If a clustering algorithm (like K-means or EM) incorrectly splits a single cluster into two, it creates a "suspicious" alignment between the resulting cluster means and their covariances. As shown in Figure 6, the group-theoretic framework can detect this failure. In simulations with 800 trials, a genericity ratio near 1 strongly correlated with clustering success (accuracy > 99%), while deviations signaled that the algorithm had failed to capture the true underlying structure.
Conclusion
While the framework provides a powerful unification of causal principles, it currently does not account for confounding variables—where an unobserved third variable causes both X and Y. Additionally, while the paper demonstrates the framework's applicability to standard models, automating the selection of the optimal group for new, complex datasets remains an open challenge. Future work aims to integrate these principles more deeply into generative models like VAEs and GANs.
Summary updated