Abstract
Distributionally robust optimization (DRO) addresses distribution shifts by optimizing over an ambiguity set of potential distributions. To balance conservatism and accuracy, these sets should leverage structural information from the nominal distribution. We propose Structural Causal Optimal Transport (SCOT), which incorporates structural equations to create more realistic ambiguity sets. We demonstrate that SCOT refines previous causal methods, overcomes the curse of dimensionality, and allows for efficient computation via a relaxed formulation. Numerical results show that SCOT significantly reduces conservatism compared to graph-only causal methods.
Introduction
Machine learning models often suffer from performance degradation when deployment data differs from training data. DRO mitigates this by considering an "ambiguity set"—a collection of probability distributions centered around the observed data. However, standard ambiguity sets often include unrealistic distributions, leading to overly conservative models. While previous work utilized causal graphs (e.g., G-causal OT), they ignored the specific functional relationships (structural equations) that define how variables interact. This paper introduces a framework to incorporate these equations to refine ambiguity sets.
Methodology: Structural Causal Optimal Transport
We introduce SCOT to prune unrealistic distributions by enforcing consistency with a set of structural equations $\mathcal{F}$. Unlike previous methods that only consider the causal graph, SCOT utilizes the full functional form of relationships. This is achieved by mapping the problem from the complex feature space to an independent "noise" (exogenous) space. This transformation simplifies the optimization and allows for efficient computation via a relaxed formulation solved with Difference-of-Convex (DC) programming and the Sinkhorn algorithm.
Key Insights: Nested Ambiguity Sets
A central contribution of this work is the theoretical proof of a hierarchy among ambiguity sets. The Structural Causal set ($\mathcal{B}^\mathcal{F}$) is a subset of the G-causal set ($\mathcal{B}^\mathcal{G}$), which in turn is a subset of the Adapted set ($\mathcal{B}^\mathcal{A}$), all contained within the classical Wasserstein set ($\mathcal{B}$).
This nesting confirms that SCOT is the most refined approach. It effectively "breaks the curse of dimensionality" because its convergence rate depends on the complexity of individual causal components rather than the total dataset dimension. Specifically, the required sample size grows with the maximum dimension of a single noise component rather than the total dimension $d$.
Results and Evaluation
Numerical evaluations using simulated causal models (e.g., Age and Education) demonstrate the advantages of SCOT over classical and G-causal approaches.
- Tighter Visual Boundaries: As seen in panels (b) and (c) above, the SCOT ambiguity set maintains causal relations, whereas classical sets expand into regions that violate the underlying causal structure.
- Reduced Conservatism: Panel (d) shows that the worst-case loss for SCOT is consistently lower than for G-causal methods. For a squared-error loss, G-causal was found to be up to 29.6% more conservative than the SCOT approach, highlighting the benefit of using structural equations.
- Finite Sample Guarantees: The method remains effective even when structural equations are estimated from data, with proven convergence to the true solution as sample size increases.
Conclusion
SCOT provides a principled way to incorporate structural causal information into DRO. By leveraging structural equations rather than just graphs, it produces tighter, more realistic ambiguity sets that avoid the curse of dimensionality. Future work will focus on generalizing the framework beyond additive noise models and establishing strong duality for the full DRO problem.
Summary updated