Many networks of sensors, clinics or survey regions each want a confidence interval for their own local parameter. Each node has plenty of unlabeled data scored by a machine-learning model but only a handful of verified labels. Prediction-powered inference (PPI) is a method that uses the model's predictions on the unlabeled data and then corrects their bias with the few labels. It gives valid intervals, but when each node has only 20 labels the correction term is noisy and the intervals stay wide. Separately, a line of signal-processing work shows that when the stacked vector of all nodes' parameters is known to lie in a low-dimensional linear subspace, projecting estimates onto that subspace filters out noise. Examples of such subspaces are conservation laws, "regional means must average to the national mean," or a shared latent component plus local offsets. Nodes can do this projection by averaging with graph neighbors, with no central server.
This note proposes to combine the two. Each node computes its own power-tuned PPI estimate and its covariance. The network then applies a covariance-weighted projection onto the known subspace, exchanging messages only with neighbors, and reports intervals whose width scales with the subspace dimension rather than the full number of node parameters. Two things are new. The first is that this purely linear pooling provably tightens intervals, even though a recent theorem says linear cross-task borrowing cannot help PPI. The second is a decentralized implementation whose intervals remain valid after a finite number of communication rounds, together with a built-in test that flags when the assumed subspace is wrong.
Core Claim
The key observation reconciles two results that look like they conflict. Prediction-Powered Inference Across Many Tasks for AI Evaluation proves a "nonlinear necessity" theorem: sharing a linear map from proxy scores to truth across tasks gives no asymptotic gain over per-task power-tuned PPI. In that setting, each task's estimand is otherwise unconstrained. A subspace constraint is a different kind of information. It is information about the estimands themselves, not about how the predictor errs.
Let $\hat{\boldsymbol w}$ denote the stacked vector of per-node PPI++ estimates, with block-diagonal asymptotic covariance $\Sigma$ (blocks are independent when node data are independent). Let $\mathcal{U}$ be an $Nd \times r$ basis of the known subspace, where $N$ is the number of nodes, $d$ the per-node parameter dimension, and $r$ the subspace dimension. The projected estimator is
$$\tilde{\boldsymbol w} = \mathcal{U}(\mathcal{U}^T\Sigma^{-1}\mathcal{U})^{-1}\mathcal{U}^T\Sigma^{-1}\hat{\boldsymbol w}, \qquad \mathrm{Cov}(\tilde{\boldsymbol w}) = \mathcal{U}(\mathcal{U}^T\Sigma^{-1}\mathcal{U})^{-1}\mathcal{U}^T \preceq \Sigma .$$
The covariance inequality is the standard generalized-least-squares fact: among linear unbiased combinations that respect the constraint, this one has minimal covariance. When noise is roughly isotropic, the total variance shrinks by about a factor $Nd/r$, because only $r$ of the $Nd$ noise directions survive. Consensus is the special case $r=d$, where the factor is $N$, matching the variance-reduction factor described in Multitask Learning Over Graphs.
The difference from the subspace-diffusion literature is the weighting. Adaptation and Learning Over Networks Under Subspace Constraints, Part II targets the orthogonal projector for point estimation. For inference with heterogeneous nodes, the right projector is oblique and weighted by $\Sigma$. Nodes with better predictors or more labels have much smaller PPI variance and should dominate the projection.
Research Program
The first component is a one-shot estimate-then-project protocol, which is simpler to make valid than streaming adaptation. Each node runs PPI++ from PPI++ on its local labeled and unlabeled data. It chooses its own power-tuning weight (how much to trust the predictions) and forms a local sandwich covariance estimate. Nodes then run $T$ rounds of neighbor averaging with a sparse combination matrix whose powers converge to the weighted projector above. The weighting enters by exchanging $\Sigma_k^{-1}\hat w_k$ and $\Sigma_k^{-1}$ terms, in the spirit of information-form averaging.
We will study whether the algebraic conditions and the Douglas-Rachford design procedure from Part I and Part II extend from orthogonal to data-dependent oblique projectors. The residual after $T$ rounds contracts geometrically at a rate set by the second-largest eigenvalue of the combination matrix. We will bound it explicitly and add the bound to the interval half-width. This makes coverage honest at finite communication, which the diffusion papers never needed because they report only mean-square deviation.
For local linear equality constraints, such as flow balance or hierarchical aggregation, we reuse the node-expansion construction of Diffusion LMS for Multitask Problems With Local Linear Equality Constraints. This avoids ever forming $\mathcal{U}$ globally.
The second component is theory. We will prove asymptotic normality and coverage of $\tilde{\boldsymbol w}$ when both the covariance and the tuning weights are estimated. We will give the exact relative efficiency against per-node PPI++, and state formally why this does not contradict the nonlinear necessity theorem: the theorem's model has no constraint on the vector of task means. We will also characterize when joint power tuning, which minimizes the projected covariance rather than each node's own, beats per-node tuning. With independent data and Loewner-optimal matrix tuning, joint tuning should add nothing. When nodes share unlabeled pools, cross-node covariance appears and joint tuning can matter. We will report this honestly rather than claim a gain that does not exist.
The third component is a misspecification check. The component of $\hat{\boldsymbol w}$ orthogonal to the subspace, measured in the $\Sigma^{-1}$ metric, is asymptotically chi-squared with $Nd - r$ degrees of freedom when the subspace is correct. It can be computed with the same neighbor exchanges. If the test rejects, nodes fall back to per-node PPI++. This addresses the known failure mode in Diffusion LMS Over Multitask Networks: pooling under a wrong structure is worse than not cooperating.
A streaming variant is a secondary aim and will be pursued only if the one-shot version works. It would inject prediction control variates into adapt-then-project diffusion, following PPI-SVRG.
Baselines
The naive reference points are classical per-node Wald intervals using only labeled data, and imputation, meaning subspace-constrained estimation that treats predictions as truth. Imputation is included to demonstrate the undercoverage documented in Prediction-Powered Inference. That paper reports that PPI needed 316 labeled examples to reject a null on the AlphaFold proteomics benchmark, against 799 for the classical approach.
Per-node PPI++ is the first serious competitor. It is exactly what the nonlinear necessity theorem says linear borrowing cannot beat. It will be run with each node's optimal tuning weight on identical splits.
GREPPI and AREPPI from Prediction-Powered Inference Across Many Tasks are the strongest existing method for scarce per-task labels. They will be run centrally, with their leave-one-task-out isotonic recalibration, giving them strictly more access to data than our decentralized method. That paper reports "substantially reduced CI widths compared to standard PPI when labels were scarce (e.g., n=20 or 40)." I found no reported numeric interval width on Lacuna for GREPPI. We will therefore reproduce it on their election benchmark before comparing.
Fed-PPI from Federated Prediction-Powered Inference is the decentralized baseline. It forces a single consensus parameter; no reported numeric interval width was found on Lacuna for it.
The upper bound is a centralized oracle: PPI++ applied to the reparametrized low-dimensional parameter $z$ with $\boldsymbol w = \mathcal U z$.
Two ablations isolate the proposal's ingredients. Unweighted (orthogonal) projection isolates the value of covariance weighting. A finite-$T$ run without the geometric residual correction isolates the value of the communication-aware interval.
"Beating" per-node PPI++ and GREPPI means achieving nominal 90% coverage (within Monte Carlo error) with smaller median interval width. "Matching" the oracle means being within 5% of its width at a moderate number of rounds $T$.
Why not just…?
Why not just reparametrize $w=\mathcal U z$ and run ordinary PPI++ at a fusion center? This is the real threat, and it is the oracle we compare against. If data could be centralized, it would be the right answer. Our contributions over it are threefold: an efficiency characterization that reconciles it with the nonlinear necessity theorem; a peer-to-peer version for settings where labels are siloed, as motivated in Fed-PPI and Privacy-Preserving Distributed Projection LMS; and finite-round validity. If the decentralized version cannot get close to the oracle's width with modest communication, the practical case collapses. The experiments measure exactly this.
Why not just use GREPPI or AREPPI across nodes? They borrow strength through the proxy-to-truth map and are blind to constraints on the estimands. The two mechanisms are complementary. We therefore also test GREPPI recalibration followed by our projection. If GREPPI alone matches projection on constrained problems, the structural claim fails.
Why not just run Fed-PPI or consensus? Consensus is the $r=d$ special case. On heterogeneous nodes it is biased, and the bias is expected to show up as undercoverage that grows with task spread. We will sweep that spread.
Proposed Experiments
The first testbed is the 50-node synthetic subspace network used in Part II and in Quantization for Decentralized Learning Under Subspace Constraints. We add synthetic proxy predictions whose accuracy we control per node, and vary per-node labels over $n_k \in \{10,20,40\}$ and the subspace ratio $r/(Nd)$. The decisive check is whether the measured width ratio against per-node PPI++ follows the predicted $\sqrt{r/(Nd)}$ scaling, and whether weighting helps most when node quality is heterogeneous.
The second testbed is the 10-node, 15-link flow network from the local-equality-constraint paper, with flow balance as the constraint.
The third testbed is a semi-real hierarchical census problem using Folktables/ACS. Nodes are geographic subregions plus a parent node with separate samples, and the exact linear constraint is that the parent mean equals the population-weighted average of the subregion means. We will run it on a random mesh graph.
The most important test uses the 2024 election benchmark from the many-tasks paper. We hypothesize an additive structure over tasks, with model effect plus prompt effect forming a linear subspace. We run the orthogonal-residual test first. We then compare projected PPI++ against GREPPI and AREPPI at $n=20$ and $40$ for coverage, interval width and mean squared error.
The idea is falsified if either of two things happens. Projection may fail to narrow intervals relative to per-node PPI++ on problems where the constraint truly holds. Or the misspecification test may fail to rescue coverage on the election tasks when additivity is false.
References
- Prediction-Powered Inference Across Many Tasks for AI Evaluation
- Multitask Learning Over Graphs: An Approach for Distributed, Streaming Machine Learning
- Adaptation and Learning Over Networks Under Subspace Constraints - Part II: Performance Analysis
- PPI++: Efficient Prediction-Powered Inference
- Adaptation and Learning Over Networks Under Subspace Constraints - Part I: Stability Analysis
- Diffusion LMS for Multitask Problems With Local Linear Equality Constraints
- Diffusion LMS over Multitask Networks
- PPI-SVRG: Unifying Prediction-Powered Inference and Variance Reduction for Semi-Supervised Optimization
- Prediction-powered inference
- Federated Prediction-Powered Inference from Decentralized Data
- Privacy-Preserving Distributed Projection LMS for Linear Multitask Networks
- Quantization for Decentralized Learning Under Subspace Constraints