Bridging Causality and Learning: How Do They Benefit from Each Other?

Mingming Gong

Introduction

Modern machine learning has achieved remarkable success in identifying complex statistical correlations—recognizing that $A$ and $B$ frequently co-occur. However, these models often struggle when faced with two fundamental challenges: intervention and distribution shift. Standard ML cannot easily predict the outcome of a system change (e.g., "How will a policy change affect outcomes?") and often fails when the training environment differs from the deployment environment.

Causal inference offers a solution by uncovering the underlying mechanisms of a system. Traditionally, discovering these causal "rules" required expensive randomized controlled trials. This research explores how we can bridge the gap between these fields, using ML to discover causal structures from observational data and using those structures to build more robust, generalizable models.

Causal Discovery from Low-Resolution Time Series

A major hurdle in causal discovery is dealing with "messy" time series data, particularly when the recording frequency is lower than the frequency of the underlying causal process. For instance, a physical process might change every second, but sensors only record data every minute. Traditional methods like Granger causality often fail in these undersampled scenarios.

To solve this, the research utilizes Vector Autoregressive (VAR) models but introduces a critical innovation: the assumption of non-Gaussian noise. While Gaussian distributions often lead to "identifiability" problems—where multiple causal graphs appear statistically identical—non-Gaussianity provides a unique "signal" that allows for the identification of the true high-resolution causal transition matrix. This discovery is implemented through unsupervised learning techniques, including variational inference and adversarial learning.

Causal Adaptive Learning (Domain Adaptation)

When a model must generalize to a new, unseen environment, it faces the problem of domain adaptation. The research proposes Augmented Directed Acyclic Graphs (DAGs) as a mathematically rigorous way to handle these shifts.

An augmented causal graph where $\theta$ nodes represent domain-specific variations in the causal mechanism.

As illustrated in the figure above, special latent variables ($\theta$) are integrated into the causal graph. These $\theta$ nodes represent the "domain" or "environment." If a variable's distribution changes across different domains, it is connected to a corresponding $\theta$ node. This structure allows the model to factorize the joint distribution into stable, invariant parts and parts that are subject to change. Domain adaptation is then treated as an inference problem: the model learns which causal mechanisms are invariant across source domains and transfers this knowledge to the target domain, ensuring robust performance even under distribution shifts.

Key Insights and Results

Several transformative ideas emerge from this synthesis of causality and machine learning:

  • The Blessing of Non-Gaussianity: In many statistical contexts, non-Gaussian noise is viewed as a complication. Here, it is a feature that breaks data symmetry, allowing researchers to determine the direction of causality that would otherwise be ambiguous under a bell curve.
  • Causality as a Generalization Tool: Rather than just explaining the past, causality is framed as a tool for predicting future failure points. By identifying stable causal links across environments, we can build models that do not require constant retraining.
  • Resolution Awareness: The research warns that causal relationships discovered at one time scale (e.g., daily stock prices) may be misleading if the true mechanism operates at a different scale (e.g., millisecond trading).

Theoretical proofs demonstrate that causal relations can indeed be recovered from subsampled data under non-Gaussian conditions—a problem previously considered "hopeless." Furthermore, practical algorithms using variational inference and adversarial learning have been developed to perform this discovery on real-world datasets, outperforming traditional "black-box" approaches by injecting causal structure.

Limitations and Future Directions

While promising, these methods rely on certain "mild conditions" regarding noise and transition matrices that may not hold in highly chaotic systems. Additionally, causal discovery remains computationally intensive as the number of variables grows. Future research will focus on scaling these algorithms and exploring how causal awareness can benefit other learning paradigms beyond domain adaptation, moving toward a more robust and interpretable form of artificial intelligence.

Summary updated

Abstract

Modern machine learning techniques can discover complicated statistical dependencies between ran- dom variables, usually in the form a statistical model, and make use of these dependencies to per- form predictions on future observations. How- ever, many real problems involve causal inference, which aims to infer how the data generating sys- tem should behave under changing conditions. To perform causal inference, we need not only statisti- cal dependencies but also causal structures to deter- mine the system’s behavior under external interven- tions. In this paper, I will be focusing on two essen- tial problems that bridge causality and learning and investigate how they can benefit from each other. On the one hand, since conducting randomized controlled experiments for causal structure discov- ery is often expensive or infeasible, it would be valuable to investigate how we can explore modern machine learning algorithms to search for causal structures from observational data. On the other hand, since causal structure provides information about the distribution changing properties, it can be used as a fundamental tool to tackle a major chal- lenge for machine learning: the capability of gener- alization to new distributions and prediction in non- stationary environment.
BibTeX
@inproceedings{gong2020bridging,
  author = {Mingming Gong},
  title = {{Bridging Causality and Learning: How Do They Benefit from Each Other?}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/725},
}
Figure