Dis-entangling Mixture of Interventions on a Causal Bayesian Network Using Aggregate Observations

Gaurav Sinha, Ayush Chauhan, Aurghya Maiti, Naman Poddar, Pulkit GoelarXiv 2019 · OpenReview

Abstract

We study the problem of separating a mixture of distributions, all of which come from interventions on a known causal bayesian network. Given oracle access to marginals of all distributions resulting from interventions on the network, and estimates of marginals from the mixture distribution, we want to recover the mixing proportions of different mixture components. We show that in the worst case, mixing proportions cannot be identified using marginals only. If exact marginals of the mixture distribution were known, under a simple assumption of excluding a few distributions from the mixture, we show that the mixing proportions become identifiable. Our identifiability proof is constructive and gives an efficient algorithm recovering the mixing proportions exactly. When exact marginals are not available, we design an optimization framework to estimate the mixing proportions. Our problem is motivated from a real-world scenario of an e-commerce business, where multiple interventions occur at a given time, leading to deviations in expected metrics. We conduct experiments on the well known publicly available ALARM network and on a proprietary dataset from a large e-commerce company validating the performance of our method.

Introduction

In complex systems like e-commerce platforms, multiple changes—such as UI updates, backend algorithm tweaks, and marketing campaigns—often occur simultaneously. Each of these represents an "intervention" on the system's causal structure. Analysts frequently observe shifts in high-level business metrics (marginals) but lack individual user-level data to attribute these changes. This paper addresses the "credit/blame attribution" problem: identifying which intervention caused how much of an observed shift in aggregate metrics. Mathematically, this is framed as separating a mixture of distributions where each component corresponds to a specific intervention on a Causal Bayesian Network (CBN).

Methodology

The core challenge is identifiability: determining if mixing proportions can be uniquely recovered from aggregate results. The authors prove that while general identifiability is impossible using only marginals, it is achievable under Assumption 1: for every variable in the network, there must be at least one state (category) that was not targeted by any intervention in the current mixture.

Constructive Algorithm and DIMM

For scenarios with exact marginals, the authors provide a constructive algorithm based on a linear system to recover proportions exactly.

Linear system for recovering mixing proportions

In real-world settings where data is noisy, they propose the Deviations in Marginals Method (DIMM). This optimization framework uses Sequential Quadratic Programming (SLSQP) to minimize the discrepancy between observed marginals and those predicted by the mixture model, adhering to causal constraints and the "untouched category" assumption.

Experimental Evaluation

The method was evaluated on two datasets:

  1. ALARM Network: A standard Bayesian network for ICU monitoring (37 nodes, 46 edges) used to validate accuracy against ground truth.
  2. Proprietary E-commerce Data: Real-world click-stream data with 32 features and 1,122 categories, where the causal graph was learned via the Tetrad tool.

Key Results

On the ALARM network, DIMM demonstrated exceptional precision, recovering mixing proportions with Mean Squared Error (MSE) in the range of $10^{-5}$ to $10^{-7}$.

Performance metrics on the ALARM network

Furthermore, DIMM proved to be over 100 times faster than the traditional Expectation-Maximization (EM) algorithm. In real-world e-commerce tests, DIMM identified 54 active interventions with 88% accuracy, despite only having access to aggregate data, whereas EM requires full joint distributions.

Discussion and Future Work

By utilizing the Causal BN as a "map," the DIMM method significantly reduces the data requirements for distribution unmixing. The "untouched category" assumption is particularly well-suited for business contexts, where interventions rarely target every possible value of a feature (e.g., a test might target Chrome users but leave Firefox users as a control).

Currently, the model is limited to "perfect" interventions (fixing a variable to a single value). Future research aims to extend this to "soft" interventions or mechanism changes and to improve robustness in cases where the underlying Directed Acyclic Graph (DAG) structure is only partially known.

Summary updated

Abstract

We study the problem of separating a mixture of distributions, all of which come from interventions on a known causal bayesian network. Given oracle access to marginals of all distributions resulting from interventions on the network, and estimates of marginals from the mixture distribution, we want to recover the mixing proportions of different mixture components. We show that in the worst case, mixing proportions cannot be identified using marginals only. If exact marginals of the mixture distribution were known, under a simple assumption of excluding a few distributions from the mixture, we show that the mixing proportions become identifiable. Our identifiability proof is constructive and gives an efficient algorithm recovering the mixing proportions exactly. When exact marginals are not available, we design an optimization framework to estimate the mixing proportions. Our problem is motivated from a real-world scenario of an e-commerce business, where multiple interventions occur at a given time, leading to deviations in expected metrics. We conduct experiments on the well known publicly available ALARM network and on a proprietary dataset from a large e-commerce company validating the performance of our method.
BibTeX
@misc{sinha2019dis,
  author = {Gaurav Sinha and Ayush Chauhan and Aurghya Maiti and Naman Poddar and Pulkit Goel},
  title = {{Dis-entangling Mixture of Interventions on a Causal Bayesian Network Using Aggregate Observations}},
  year = {2019},
  eprint = {1912.00163},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1912.00163},
}
Figure