Asynchronous Distributed Gaussian Process Regression

Zewen Yang, Xiaobing Dai, Sandra HircheAAAI 2025 · DOI

Abstract

Gaussian Processes (GPs) are the gold standard for Bayesian regression due to their mathematically grounded uncertainty measures. However, their cubic scaling with data size necessitates Distributed GPs (DGPs), where data is partitioned among parallel experts. In practical settings like edge computing or robotics, these experts often operate asynchronously, leading to "stale" predictions. This paper presents AsyncDGP, the first framework designed to handle such asynchrony by intelligently weighting predictions based on both posterior variance and spatial distance from the current state. We provide theoretical error bounds and demonstrate that AsyncDGP outperforms existing DGP aggregation methods across multiple benchmarks.

Introduction: The Challenge of Asynchrony

Distributed Gaussian Processes typically assume that all experts provide predictions for the same query point simultaneously. In real-world applications—such as a robot navigating a dynamic environment—computational nodes have varying speeds. By the time an expert finishes processing a prediction for a state $x(t_0)$, the system has already moved to $x(t_1)$. Standard aggregation methods like Product of Experts (PoE) or Bayesian Committee Machines (BCM) fail in these scenarios because they cannot account for the "spatial staleness" of delayed information.

Methodology: The AsyncDGP Framework

AsyncDGP addresses asynchrony by treating it as a trade-off between predictive accuracy and query proximity. The core of the approach is a novel performance criterion, $\eta$, which augments the standard GP posterior variance with a bias term proportional to the distance between the original query point and the current system state.

System architecture of AsyncDGP showing the interaction between the plant, central server, and distributed GP models.

As shown in the architecture diagram above, the system utilizes a Central Server to manage an "Information Set" buffer. This buffer stores recent predictions from various experts. As new predictions arrive, the server performs:

  1. Filtering: Predictions that are less reliable than the GP's prior mean are discarded.
  2. Asynchronous Aggregation: Weights are assigned to buffered predictions based on their $\eta$ values. Stale or highly uncertain predictions receive lower weights, while the prior mean acts as a "safety net" baseline.
  3. Control Integration: The framework is integrated into a closed-loop control law, ensuring that tracking errors remain bounded even under varying computational delays.

Experimental Evaluation

The framework was rigorously tested on three standard regression benchmarks (KIN40K, SARCOS, and PUMADYN32NM) and a custom simulation platform, GPgym, designed for networked online learning.

Standardized Mean Squared Error (SMSE) comparison across KIN40K, SARCOS, and PUMADYN32NM datasets, showing AsyncDGP's superior performance over iterations.

As illustrated in the performance curves, AsyncDGP (represented by the dashed blue line) consistently achieves a lower Standardized Mean Squared Error (SMSE) compared to traditional methods like rBCM, gPOE, and MOE across all datasets. A key finding is the benefit of the "Information Set": increasing the buffer size ($\bar{J}$) allows the system to leverage older predictions that remain spatially relevant, further improving accuracy.

Key Insights and Conclusion

The primary innovation of AsyncDGP is the use of spatial distance as a proxy for time. By leveraging the Lipschitz continuity of GP kernels, the authors quantify the "decay" of a prediction as the system moves. This allows for deterministic error guarantees, which are critical for safety-sensitive robotics applications. While AsyncDGP shows remarkable robustness, future work could address memory management in extremely high-dimensional spaces and explore fully decentralized architectures. In summary, AsyncDGP provides a theoretically sound and practically effective solution for real-time, distributed Bayesian learning in asynchronous environments.

Summary updated

BibTeX
@inproceedings{yang2025asynchronous,
  author = {Zewen Yang and Xiaobing Dai and Sandra Hirche},
  title = {{Asynchronous Distributed Gaussian Process Regression}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i21.34359},
}
Figure