A Gaussian process (GP) is a Bayesian regression model that predicts a value together with an error bar. Large spatial datasets increasingly live on several machines, for example regional servers that each hold the sensor readings of their own area. The fast way to run a GP at scale is to predict each query point from only its nearest data points. Once the data is split across machines, though, a query near a regional border has neighbors on two machines. Ignoring the other machine produces visible jumps in the prediction at the border. Gathering every query's true neighbors from every machine floods the network.
Wireless engineers have faced exactly this problem with access points and users in "cell-free" networks. Their answer is to serve users in the middle of a region locally, at no communication cost. Only users near a border get a small, overlapping, user-specific group of access points that spans at most two regional processors. Antennas that contribute little are pruned. We propose to carry this center/edge rule over to distributed GP inference. Each query is answered by its home machine, unless a cheap, kernel-aware test says that remote data would change the answer by more than a set fraction. In that case the machine fetches a few neighbors from at most one or two adjacent machines, within a fixed per-link communication budget.
Core Claim
Our claim is that, for spatially sharded data, a per-query center/edge decision based on kernel covariance gets close to the accuracy and calibration of centralized nearest-neighbor GP prediction. It should also remove the border discontinuities of disjoint local experts, while communicating only a small fraction of the bytes needed to gather exact global neighbors for every query. The method should beat fixed "halo" replication of border data when length-scales vary across space or when data streams in. It is not expected to win on static data with a single uniform length-scale.
The wireless side supplies the mechanism. Hybrid Network- and User-Centric Scalable Cell-Free Massive MIMO for Fronthaul Signaling Minimization introduces HybridUA. It splits users into cluster-center users, served by one processor's access points, and cluster-edge users, served across up to two processors. It keeps only the access points that together supply 95% of the channel gain. The paper reports cutting inter-processor signaling by 71% to 94% relative to the earlier SCF1lim scheme, with spectral-efficiency losses typically under 9%. Sparse Beamforming and User-Centric Clustering for Downlink Cloud Radio Access Network motivates treating cluster size as a quantity bounded by link capacity rather than chosen freely. Our translation replaces channel gain with covariance between the query and data points, and replaces fronthaul capacity with a byte budget between machines.
The GP side supplies the inference engine and the tension we must resolve. Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression introduces GPnn, which predicts from the $m$ nearest neighbors and estimates hyperparameters on a small subset. It reports training on $n = 1.6\times10^6$ HouseElectric points in about 30 seconds on a laptop, with better MSE and NLL than SVGP on most UCI sets. GPnn is centralized and fixes $m$ in advance. MuyGPs supplies local leave-one-out hyperparameter fitting, whose cost does not depend on $n$; it, too, treats each query independently on one machine. Local and global sparse Gaussian process approximations proposes PIC, which splits data into disjoint blocks and links them through global inducing points. PIC is the closest existing combination of local and global structure. Its blocks are fixed and non-overlapping. Ours are formed per query and overlap, and no global inducing set has to be kept in sync across machines.
Research Program
Data are partitioned spatially across $P$ workers, and each worker builds its own neighbor index. Each worker also broadcasts a small summary of its border: a few hundred representative points sampled near its region's boundary, refreshed periodically. A query is routed to the worker that owns its location. That worker first finds the $m$ nearest local points. It then uses the border summaries to compute an upper bound on how much remote data could matter, defined as
$$\rho(q) = \frac{\sum_{r \in R_q} k(q,r)^2}{\sum_{j \in L_q} k(q,j)^2 + \sum_{r \in R_q} k(q,r)^2},$$
where $q$ is the query, $k$ is the fitted kernel, $L_q$ is the local neighbor set, and $R_q$ is the set of border representatives from adjacent workers that lie closer to $q$ than the $m$-th local neighbor.
If $\rho(q)$ falls below a threshold $\tau$, the query is a "center" query and is answered locally. The threshold $\tau = 0.05$ mirrors HybridUA's 95% gain rule. Otherwise the query is an "edge" query. The worker requests a few remote neighbors from the one or two adjacent workers with the largest share of $\rho(q)$, merges them with $L_q$, and solves the usual small $m \times m$ GP system.
When many edge queries compete for one link, a scheduler spends the per-link byte budget on the queries with the largest expected reduction in predictive variance. The rest are served locally, and the system records that they were served without remote help.
Because the kernel enters the test, the effective border width adapts automatically. It is wide where the fitted length-scale is long, narrow where it is short, and follows anisotropic length-scales. A fixed halo cannot do this. Hyperparameters are fitted locally on each worker with MuyGPs-style leave-one-out estimation, starting from a shared initial value.
Calibration needs specific attention, because the uncertainty calibration step in GPnn, which scales the noise variance after fitting, assumes all neighbor sets are formed the same way. We will fit separate calibration scales for center queries, fetched edge queries, and budget-denied edge queries. We will then test whether coverage holds within each group, not just on average.
A second study addresses Gap 3 in the file. GPnn says local prediction is robust to misspecified hyperparameters as $n$ grows. Whom to Trust? Elective Learning for Distributed Gaussian Process Regression shows that fusing agents with bad priors wrecks accuracy. It reports a total average error of 22.37 for its Pri-GP method versus 79.58 for product of experts on a 4-agent sine task. We will corrupt the hyperparameters of some workers and measure whether merging raw neighbor data, which our method does, is less fragile than fusing whole predictions, which product-of-experts methods do. Our hypothesis is that sharing data points rather than posteriors inherits GPnn's robustness, so no trust filter is needed. If that hypothesis fails, a Pri-GP-style filter on remote contributions would be added.
Baselines
The do-nothing reference is disjoint local GPnn. Each worker answers its own queries from its own data with zero communication. This sets the communication floor and shows how large the border errors are.
The accuracy ceiling is centralized GPnn (Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression) on the pooled data. For context, its reported HouseElectric training time is about 30 seconds. On MODIS, MuyGPs (MuyGPs) reports an RMSE between 1.53 and 1.55 with 50 neighbors, in under a minute on a laptop, against about 120 minutes for the SPDE method.
The strongest distributed competitor is the analogue of SCF1lim: every query gathers its exact global $m$ nearest neighbors through a distributed neighbor search. Its accuracy matches the centralized ceiling, and its communication is the highest.
The strongest cheap competitor is halo replication. Each worker permanently copies all foreign data within a border width $w$. For fairness, $w$ is tuned on validation data with the same tuning budget we give $\tau$, and its memory and synchronization bytes are counted.
We also run three prediction-fusion methods: product of experts, the Bayesian committee machine, and Pri-GP (Whom to Trust? Elective Learning for Distributed Gaussian Process Regression). These fuse whole-shard predictions instead of sharing points.
We also run PIC (Local and global sparse Gaussian process approximations) with shard-aligned blocks. For PIC on kin-40k, the file reports only qualitative gains over global sparse GPs: no reported result found on Lacuna for PIC's numeric MSE or NLPD. We will therefore rerun it ourselves.
All methods use the same partition, the same $m$, the same hyperparameter-fitting routine, and the same hardware. Wall-clock time and bytes moved are logged under identical message serialization.
Two ablations isolate the key ingredient. The first replaces the kernel-aware test $\rho(q)$ with a purely geometric rule, "fetch if the distance to the border is less than the $m$-th neighbor distance". The second removes the budget-aware scheduler.
We count the method as beating the field if two conditions hold. Its NLL must be within 2% of centralized GPnn while it moves at most 20% of the bytes of exact global gathering. And it must beat a tuned halo on the combined accuracy/bytes curve in at least the non-stationary and streaming settings.
Why not just…?
Why not just replicate a halo of border data on each worker? This is the most serious threat. On static data with one global length-scale, a well-tuned halo may match us, and we expect to report that honestly. Halo width is a single number, so it must be set for the longest length-scale anywhere, which wastes memory where correlations are short. Every streaming insert near a border must also be pushed to neighbors whether or not any query needs it. We will settle the question in three settings: stationary data, data whose length-scale varies across space, and streaming data. Our claim survives only if we win in the latter two.
Why not just centralize the data? GPnn already handles 1.6 million points on a laptop, so for static, moderately sized data, centralizing is simpler. Our setting is data that is generated on different machines and keeps arriving. There, centralizing costs bytes in proportion to the data arrival rate, while our scheme costs bytes in proportion to the query rate times the fraction of edge queries. We will plot both against the ratio of queries to new data and state where the crossover lies, rather than claiming a universal win.
Why not just fuse each worker's predictions (product of experts, committee machine, Pri-GP)? Fusion costs one message per query per worker. It does not repair border discontinuities, because each expert is still blind to the others' raw data. The Pri-GP results show fusion is fragile when priors are bad: product-of-experts error of 79.58 versus 22.37 for Pri-GP. Our corrupted-hyperparameter study tests directly whether sharing points avoids that fragility.
Why not just use a global sparse GP with inducing points? Approximation-Aware Bayesian Optimization notes that a fixed inducing budget smooths over sharp local features. GPnn reports beating SVGP in MSE and NLL on most UCI sets. A distributed inducing model would also need inducing points kept in sync across all workers. SVGP is still included as a reference on the UCI sets.
Proposed Experiments
Experiment 1: MODIS land surface temperature. The data (about 105,000 training and 42,000 test points on a 500×300 grid) is split into 4, 9, and 16 spatial tiles. We report RMSE, NLL, 90% and 95% interval coverage, and bytes per query. We also report a border-jump statistic: the mean absolute change in predictive mean between adjacent grid cells that straddle tile borders, minus the same quantity for interior pairs. The MuyGPs RMSE of 1.53–1.55 is the reference for centralized local prediction.
Experiment 2: large UCI datasets. HouseElectric, Road3D, and Song are partitioned by k-means on the input space, which is a stress test because borders in high dimensions are fuzzy. We report MSE, NLL, and wall-clock time against centralized GPnn.
Experiment 3: non-stationary and streaming. We build a synthetic spatial field whose length-scale varies tenfold across the domain. We then replay MODIS observations as a stream across tiles. This is the experiment that decides the halo comparison.
Experiment 4: a direct test of the source analogy. We simulate a multi-processor cell-free layout in which each processor holds large-scale channel-gain measurements from its own access points, indexed by user position. The distributed GP predicts gains at new user positions so that clusters can be formed. We measure gain-prediction NLL and inter-processor bytes. We also measure downstream 5%-outage spectral efficiency when HybridUA clustering uses predicted gains rather than measured ones, against HybridUA's reported 71–94% signaling reduction.
Decisive checks. First, does the per-group calibrated coverage of edge queries stay within two points of nominal? Second, does the kernel-aware test beat the geometric ablation under varying length-scales? If neither holds, the contribution reduces to a halo with extra steps, and we will say so.
References
- Hybrid Network- and User-Centric Scalable Cell-Free Massive MIMO for Fronthaul Signaling Minimization
- Sparse Beamforming and User-Centric Clustering for Downlink Cloud Radio Access Network
- Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression
- MuyGPs: Scalable Gaussian Process Hyperparameter Estimation Using Local Cross-Validation
- Local and global sparse Gaussian process approximations
- Whom to Trust? Elective Learning for Distributed Gaussian Process Regression
- Approximation-Aware Bayesian Optimization