Calibrated Functional Prediction from Overlapping Nonlinear Observations

Peida LiarXiv 2026 · arXiv

In the world of machine learning, we often treat tasks as independent. But what if multiple "audits"—nonlinear queries or functions—are performed on the same hidden dataset? If these audits overlap, sharing rows and underlying structures, we should be able to use the results of one to better predict the outcome of another.

A new paper by Peida Li from Peking University, "Calibrated Functional Prediction from Overlapping Nonlinear Observations," introduces a rigorous framework for doing exactly this. It combines kernel methods, Gaussian projections, and high-precision numerical certification to provide not just predictions, but guarantees.

TL;DR

The paper presents a method to predict the outcomes of nonlinear functions (audits) that share a hidden underlying state. By projecting these functions into a common Gaussian space and using "residual samples" to capture what's left over, the authors achieve state-of-the-art accuracy with a 15x speedup in numerical certification compared to standard high-precision libraries.

What Problem Does This Paper Address?

Imagine a "hidden table" of data. Different auditors come along and ask different nonlinear questions: "What is the retraining regret if we drop this feature?" or "What is the variance of this specific subset?" Because these auditors are looking at the same hidden rows, their observations are correlated.

The challenge is twofold:

  1. Complexity: Nonlinear functions are hard to model. A global quadratic expansion of a large table leads to a "curse of dimensionality" (a quadratic number of coordinates).
  2. Trust: In auditing, a "maybe" isn't good enough. We need to know exactly how much numerical error exists in our integration and rounding, and we need a statistical guarantee that our prediction interval actually contains the truth.

Existing tools like Gaussian Processes or Polynomial Chaos filtering address parts of this, but they often struggle with overlapping "bags" of data or fail to provide deterministic certificates for the final numerical output.

Methods

The authors propose a multi-stage pipeline to bridge the gap between raw observations and certified predictions:

  • Local-to-Global Kernel: Instead of modeling the whole table at once, they compute a "local projection" for each operator. They then "lift" these projections isometrically into a common Gaussian space. This creates a sparse kernel that naturally handles repeated row identities and cross-row dependencies.
  • Residual Covariance: Since a degree-two projection doesn't capture all nonlinear variation, they extend the kernel using "shared-state residual samples." This is essentially a smart way to use Monte Carlo samples to fill in the functional gaps.
  • Numerical Certification: This is the paper's "secret sauce." They use positive matrix integration and Gamma-weighted Gauss-Radau rules to calculate bounds on the prediction. Unlike standard black-box models, this method accounts for signed coefficient errors and integration residuals.
  • Task-Level Calibration: To turn these predictions into reliable intervals, they use a calibration technique (similar to conformal prediction) that provides finite-sample coverage guarantees.

Datasets

The method was evaluated across 400 different conditions using ten benchmark populations:

  • Simulated Environments: Gaussian, Heteroscedastic (varying noise), Heavy-tailed, and Nonlinear simulations.
  • Real-World Regression Tables (UCI Repository): Concrete Compressive Strength, Airfoil Self-Noise, Combined Cycle Power Plant, Red/White Wine Quality, and Superconductivity.

The authors introduced twenty new reference contexts per dataset to ensure the results weren't just a fluke of one specific data split.

Key Insights and Ideas

  • The Power of Shared State: The core "aha" moment is that overlapping observations aren't just redundant; they are constraints on a shared hidden state. By modeling that state, you can predict a completely new nonlinear function more accurately than if you looked at that function in isolation.
  • Sparse Lifting: By using a compact row-feature map, the authors avoid the computational explosion usually associated with higher-order expansions.
  • Certification is Feasible: Often, researchers ignore numerical rounding or integration errors as "negligible." This paper proves you can bound these errors deterministically without sacrificing massive amounts of compute time.
  • Gamma Quadrature Efficiency: The use of specialized Gamma-weighted quadrature rules allowed the authors to outperform the "gold standard" Arb/Petras integration library by a factor of 15.

Key Results

  • Superior Accuracy: The method achieved a mean complete-loss ratio of 0.9911 compared to the retrospective minimum of 174 other public configurations. In plain English: it consistently outperformed a massive array of baseline models.
  • Efficiency: On 40 different profiles, the "Gamma engine" for numerical integration showed a geometric mean cold-time ratio of .0646 compared to strengthened Arb/Petras integration.
  • Tight Bounds: The numerical error upper bounds for saved predictions were remarkably tight, often in the range of $10^{-7}$ to $10^{-11}$ in original regret units.

Limitations and Future Work

  • Operator Constraints: The current work focuses on Schur-rational operators (functions that can be expressed as a ratio of polynomials of degree at most two). While broad, this doesn't cover every possible nonlinear audit.
  • Gaussian Assumption: The "working law" assumes a Gaussian distribution for the hidden state. While the method is robust, extreme non-Gaussianity in the hidden features might require more complex projection spaces.
  • Computational Scaling: While sparse, the kernel ridge regression still involves an $O(n^3)$ matrix solve, which might become a bottleneck for extremely large numbers of source observations.

Final Thought: This paper is a significant step toward "Responsible AI" in auditing. It moves us away from "black-box" heuristics and toward a world where every prediction comes with a mathematically rigorous receipt.

Summary updated

Abstract

We study prediction of nonlinear audit functionals from scalar observations of a shared hidden table state. For Schur-rational operators, local Gaussian projections yield a common sparse kernel for overlapping tasks, repeated row identities, and cross-row dependence. Shared-state residual samples extend this kernel through a positive-semidefinite feature representation. Positive matrix integration and sample-value enclosures connect the original inputs to the actual saved hybrid prediction. A sorted calibration-role perturbation bound then certifies its complete finite-pool pinball risk. Independent complete-loss selection chooses the ridge, score scale, and residual weight, while task-rank calibration supplies an upper endpoint. Evaluation uses twenty new reference contexts per dataset at two task sizes, giving 400 conditions across ten benchmark populations. The mean dataset complete-loss ratios are 0.9977 to the degree-two kernel, 0.9961 to matched complete-MC shrinkage, and 0.9911 to the retrospective minimum over 174 public configurations. The last ratio has a paired reference-bootstrap interval of [0.9898,0.9927]. On 40 profiles at coefficient tolerance $10^{-7}$, Gamma quadrature has a geometric mean cold-time ratio of .0646 to strengthened Arb/Petras integration.
BibTeX
@misc{li2026calibrated,
  author = {Peida Li},
  title = {{Calibrated Functional Prediction from Overlapping Nonlinear Observations}},
  year = {2026},
  eprint = {2611.20567},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20567},
}
Figure