Standard errors for regression on relational data with exchangeable errors.

Tyler H. McCormick, Bailey K. Fosdick, Frank W. Marrs

Abstract

Relational arrays represent interactions or associations between pairs of actors, often in varied contexts or over time. Such data appear as, for example, trade flows between countries, financial transactions between individuals, contact frequencies between school children in classrooms, and dynamic protein-protein interactions. This paper proposes and evaluates a new class of parameter standard errors for models that represent elements of a relational array as a linear function of observable covariates. Uncertainty estimates for regression coefficients must account for both heterogeneity across actors and dependence arising from relations involving the same actor. Existing estimators of parameter standard errors that recognize such relational dependence rely on estimating extremely complex, heterogeneous structure across actors. Leveraging an exchangeability assumption, we derive parsimonious standard error estimators that pool information across actors and are substantially more accurate than existing estimators in a variety of settings. This exchangeability assumption is pervasive in network and array models in the statistics literature, but not previously considered when adjusting for dependence in a regression setting with relational data. We show that our estimator is consistent and demonstrate improvements in inference through simulation and a data set involving international trade.

Introduction

In traditional regression analysis, observations are typically assumed to be independent. However, in relational data—often represented as networks or arrays—this assumption is fundamentally violated. If Country A trades with both Country B and Country C, the resulting data points are intrinsically linked by their shared connection to Country A. This "dyadic dependence" poses a significant challenge for calculating accurate uncertainty (standard errors) for regression coefficients.

Currently, researchers rely on two primary strategies. The first involves parametric latent variable models, which are accurate but computationally intensive, often requiring slow Bayesian MCMC simulations. The second involves dyadic clustering (sandwich estimators), which are computationally efficient but suffer from high variability in finite samples and are frequently "anticonservative," leading to a higher rate of false-positive results. This paper introduces a middle ground: leveraging the structural symmetry of networks to simplify error estimation without the overhead of full latent variable models.

The Exchangeability Framework

The core innovation of this work is the application of the exchangeability assumption to the error structure of relational arrays. Joint exchangeability implies that the statistical properties of the array remain unchanged if the labels of the actors are swapped.

Configurations of dyadic dependence and the resulting exchangeable covariance structure.

By recognizing this symmetry, the authors prove that a massive covariance matrix $\Omega$—which would normally contain a daunting number of parameters—collapses into a small set of unique values. For single-layer networks, this reduces to just 6 unique values; for multi-layered arrays, it is 12. This allows for parsimonious estimation, where information is pooled across symmetric configurations (as seen in the dyad configurations above) to stabilize error estimates. Instead of estimating a unique error term for every pair, the model "borrows strength" from the entire network structure.

Methods and Implementation

The authors integrate these insights into a Feasible Generalized Least Squares (FGLS) framework. This approach allows researchers to iteratively refine regression coefficients and error estimates, achieving higher efficiency than standard Ordinary Least Squares (OLS). The methods are implemented in the R package netregR, providing a practical tool for researchers to apply these robust estimators to large-scale relational datasets.

Key Results and Empirical Evaluation

The proposed estimator was validated through extensive simulations and an analysis of international trade data (58 countries over 20 years).

Estimated coverage of 95% confidence intervals for exchangeable vs. dyadic clustering estimators across binary and positive real covariates.

Superior Coverage and Efficiency

In simulations, the exchangeable estimator consistently outperformed standard dyadic clustering. As shown in the figure above, the exchangeable estimator (red) produced 95% confidence intervals that accurately contained the true value approximately 95% of the time, regardless of the number of actors. In contrast, the dyadic clustering method (blue) was systematically biased downward and exhibited much higher variability, particularly for binary variables.

Predictive Power and Speed

In the international trade study, the exchangeable FGLS approach improved predictive power ($R^2$) by approximately 10% over complex mixed-effects models and 30% over OLS. Furthermore, the method is computationally efficient, running in a fraction of the time required for Bayesian mixed-effects models, making it scalable for large networks.

Conclusion

The exchangeable estimator provides a "sweet spot" in relational data analysis. It is more robust than parametric models because it makes fewer assumptions about the specific form of the error, yet it is significantly more stable than empirical sandwich estimators because it exploits the inherent symmetry of the data. While the assumption of exchangeability may be restrictive if certain actors behave as unique "super-hubs," the authors provide permutation tests to validate the assumption, offering a powerful new tool for robust inference in network science.

Summary updated

Abstract

Relational arrays represent interactions or associations between pairs of actors, often in varied contexts or over time. Such data appear as, for example, trade flows between countries, financial transactions between individuals, contact frequencies between school children in classrooms, and dynamic protein-protein interactions. This paper proposes and evaluates a new class of parameter standard errors for models that represent elements of a relational array as a linear function of observable covariates. Uncertainty estimates for regression coefficients must account for both heterogeneity across actors and dependence arising from relations involving the same actor. Existing estimators of parameter standard errors that recognize such relational dependence rely on estimating extremely complex, heterogeneous structure across actors. Leveraging an exchangeability assumption, we derive parsimonious standard error estimators that pool information across actors and are substantially more accurate than existing estimators in a variety of settings. This exchangeability assumption is pervasive in network and array models in the statistics literature, but not previously considered when adjusting for dependence in a regression setting with relational data. We show that our estimator is consistent and demonstrate improvements in inference through simulation and a data set involving international trade.
BibTeX
@inproceedings{mccormick2017standard,
  author = {Tyler H. McCormick and Bailey K. Fosdick and Frank W. Marrs},
  title = {{Standard errors for regression on relational data with exchangeable errors.}},
  booktitle = {Biometrika},
  year = {2017},
}
Figure