Graph Neural Network based Double Machine Learning Estimator of Network Causal Effects

Seyedeh Baharan Khatami, Harsh Parikh, Haowei Chen, Sudeepa Roy, Babak SalimiarXiv 2024 · arXiv · OpenReview

TL;DR

This paper introduces a novel method for estimating causal effects in social networks, addressing the challenge of "interference" where a person's outcome is influenced by their neighbors' actions. By combining Graph Neural Networks (GNNs) with the Double Machine Learning (DML) framework, they achieve accurate and robust estimation of direct and peer effects, even with complex, high-dimensional network data. This approach is crucial for understanding interventions and behaviors in networked systems, from social media to public health.

What Problem Does This Paper Address?

Estimating causal effects in social networks is notoriously difficult due to "interference." Imagine trying to understand the impact of a new health policy: not only does the policy directly affect individuals, but it also influences their friends and family, who in turn might influence others. This "spillover" effect, or interference, makes standard causal inference methods, which assume independent observations, inadequate. Furthermore, network structures can introduce complex "confounding" factors – a person's characteristics and their neighbors' characteristics can both influence their outcome and their treatment status. Existing methods often rely on strong, simplifying assumptions about how these network-induced confounders work, which may not hold in real-world, high-dimensional networks. This paper aims to provide a more flexible and robust method for estimating both the direct impact of an intervention on an individual and the indirect impact through their network connections.

Methods

The core of this work is the integration of two powerful techniques:

  • Graph Neural Networks (GNNs): GNNs are adept at learning from data with relational structures, like social networks. They can effectively aggregate information from a unit's neighbors to create rich representations of network context. The authors specifically use the Graph Isomorphism Network (GIN) for its strong performance.
  • Double Machine Learning (DML): DML is a statistical framework designed to estimate causal effects robustly, even when the underlying models for treatment assignment (propensity score) and outcome are complex and potentially misspecified. It achieves this by using machine learning models to estimate these "nuisance" parameters and then employing a specific estimation strategy that makes the final causal estimate less sensitive to errors in those models.

The authors extend DML to the network setting by using GNNs to estimate the propensity score and outcome models. A key innovation is the use of a "focal set" – a subset of nodes whose neighborhoods do not overlap. This allows for independent estimation of nuisance parameters, which is crucial for obtaining reliable uncertainty estimates (confidence intervals) in networked settings. They also employ cross-fitting, a standard DML technique, to further improve robustness and efficiency.

Datasets

The authors evaluate their method on a combination of datasets:

  • Real-World Data:
    • Indian Village Dataset: A survey dataset from villages in India, containing individual features and 12 distinct social networks. This was used to study the impact of Self-Help Group (SHG) participation on financial risk tolerance.
  • Semi-Synthetic Data:
    • Cora, Pubmed, Flickr: These are standard academic network datasets (citation networks and image networks, respectively) where synthetic covariates, treatments, and outcomes were generated based on the network structure. This allows for ground truth evaluation.
  • Synthetic Data:
    • Stochastic Block Model (SBM): Networks generated using SBM, a model that creates communities within networks, allowing for controlled manipulation of network density and structure.

Key Insights and Ideas

  • Bridging GNNs and Causal Inference: The paper demonstrates a principled way to leverage the representational power of GNNs for causal inference in networks, overcoming limitations of previous methods that relied on hand-crafted network features or strong assumptions.
  • Doubly Robust Estimation in Networks: By combining GNNs with DML, the authors achieve a doubly robust estimator, meaning it remains consistent even if either the propensity score model or the outcome model is misspecified. This is a significant advantage for complex network data.
  • The Power of Focal Sets: The introduction of a "focal set" strategy is crucial for enabling valid uncertainty quantification in networked settings. By isolating independent units, it allows for more reliable estimation of nuisance parameters and, consequently, the causal effects.
  • Distinguishing Direct vs. Peer Effects: The framework explicitly estimates both the "average direct effect" (ADE) and the "average peer effect" (APE), providing a more nuanced understanding of how interventions propagate through networks.

Key Results

  • Superior Performance: Across three semi-synthetic datasets, their proposed method (GDML) performed on par with or better than state-of-the-art baselines in terms of mean squared error (MSE).
  • Scalability: GDML demonstrated significantly better scalability compared to many baselines, especially on larger datasets like Flickr, often completing within a 12-hour runtime limit where others failed.
  • Robustness: The method showed robustness across different network densities and data generative processes.
  • Real-World Application: In the case study on SHG participation, their method provided estimates for direct and peer effects, offering insights into financial risk tolerance. While the direct effect was positive, it was not statistically significant due to limited sample size in the focal set. The peer effect was found to be minimal.
  • Confidence Interval Coverage: The method achieved good coverage rates for 95% confidence intervals across datasets, indicating reliable uncertainty quantification.

Limitations and Future Work

  • GNN Convergence Assumptions: The framework relies on assumptions about GNN convergence rates, which are an active area of research.
  • Domain-Specific Knowledge: While accommodating high-dimensional covariates, the method still benefits from domain-specific knowledge for defining exposure mappings.
  • Densely Connected Graphs: The framework's performance can diminish with very densely connected graphs due to a reduced effective sample size in the focal set.
  • Future Directions: The authors suggest extending the framework to heterogeneous graphs, exploring higher-order graph structures, and investigating the impact of missing network ties in partially observable graphs. They also highlight the potential of using alternative graph aggregation tools beyond GNNs, such as Network Random Forests (NeRF+), for interpretability.

Summary updated

Abstract

We address the challenge of inferring causal effects in social network data. This results in challenges due to interference -- where a unit's outcome is affected by neighbors' treatments -- and network-induced confounding factors. While there is extensive literature focusing on estimating causal effects in social network setups, a majority of them make prior assumptions about the form of network-induced confounding mechanisms. Such strong assumptions are rarely likely to hold especially in high-dimensional networks. We propose a novel methodology that combines graph machine learning approaches with the double machine learning framework to enable accurate and efficient estimation of direct and peer effects using a single observational social network. We demonstrate the semiparametric efficiency of our proposed estimator under mild regularity conditions, allowing for consistent uncertainty quantification. We demonstrate that our method is accurate, robust, and scalable via an extensive simulation study. We use our method to investigate the impact of Self-Help Group participation on financial risk tolerance.
BibTeX
@misc{khatami2024graph,
  author = {Seyedeh Baharan Khatami and Harsh Parikh and Haowei Chen and Sudeepa Roy and Babak Salimi},
  title = {{Graph Neural Network based Double Machine Learning Estimator of Network Causal Effects}},
  year = {2024},
  eprint = {2403.11332},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.11332},
}
Figure