Conformally Calibrated Adaptive Computation in Neural Operators

Abdul Qadir Ibrahim, Martin Burger, Abel Maviano JonathanarXiv 2026 · arXiv

TL;DR

This paper applies conformal calibration to Fourier Neural Operators (FNOs) to adaptively select the number of Fourier modes or channel widths per input, ensuring a guaranteed error budget. The authors discover a significant "latency gap": while reducing computation theoretically saves FLOPs, actual inference speedups are often negligible or outperformed by simpler, well-calibrated static models when accounting for routing overhead.

What Problem Does This Paper Address?

Fourier Neural Operators (FNOs) are powerful tools for solving partial differential equations (PDEs), but they typically use a "one-size-fits-all" approach, applying the same spectral bandwidth and channel width to every input. However, PDE instances vary wildly in complexity; a smooth solution requires far less compute than a turbulent one.

The authors investigate whether we can adaptively "slim down" the model for simpler inputs to save time without sacrificing accuracy. Crucially, they address the latency gap: the fact that reducing theoretical operations (like truncating Fourier modes) does not always translate to faster wall-clock time on modern hardware. They also tackle the challenge of providing rigorous statistical guarantees—ensuring that the "faster" model doesn't exceed a pre-defined error budget.

Methods

The researchers use split-conformal calibration to provide a marginal guarantee on the error. They propose two main selection rules:

  1. Triangle-inequality envelope: A conservative bound that uses the disagreement between a candidate (slim) model and a full reference model to estimate error.
  2. Direct-score rule: A more efficient approach that calibrates candidate errors directly, handling cases where model error might not decrease monotonically with more compute.

They apply these rules to two types of adaptation:

  • Spectral Support ($K$): Truncating the number of Fourier modes used in the spectral layers.
  • Channel Width ($w$): Using "slimmable" training (weight sharing) to run the model at different widths (e.g., 8, 16, 24, or 32 channels).

They also distinguish between Reference-based selection (which requires running the full model first to decide which slim model to use) and Input-only routing (which uses a cheap statistic of the input, like Total Variation, to decide the compute budget).

Datasets

The study utilizes two classic PDE benchmarks:

  • Burgers’ Equation: A 1D non-linear PDE.
  • Darcy Flow: A 2D steady-state PDE representing flow through porous media.
  • New Benchmark: The authors introduced a separately generated 1,000-example Darcy test set to evaluate how well their calibration holds up on data not seen during the development of the routing policies.

Key Insights and Ideas

  • The Latency Trap: In FNOs, simply reducing the number of active Fourier modes ($K$) is often "latency-neutral." As shown in Figure 1, changing $K$ leaves the elapsed time effectively unchanged because the fixed costs of full-grid FFT transforms and pointwise channel mixing dominate the execution time.
Figure 1: Batch-1 latency across spectral support, Burgers, width 56, RTX A6000. The axis starts at zero; changing K leaves elapsed time effectively unchanged, with recorded means of 1.4934–1.5047 ms.
  • The Paradox of Reference Models: If you use a reference-based rule to save time, you've already lost. To calculate the "disagreement" with the full model, you usually have to compute the full model's prediction, negating any speedup from running a smaller candidate afterward.
  • Static Baselines are Formidable: A key "aha moment" is that a calibrated static model (e.g., always using width 16) often provides a better trade-off between speed and coverage than a complex adaptive router. Adaptive computation must be judged against these matched static comparators.
  • Total Variation as a Signal: For Darcy flow, the Total Variation (TV) of the input coefficient field is a moderate predictor of how much channel width is needed, enabling "input-only" routing that bypasses the full model pass.

Key Results

  • Mode Selection: On Burgers and Darcy, reducing spectral support provided almost no measurable latency reduction (see Table 4 for Darcy results).
Table 4: Mode-truncation latency on Darcy, standard width-32 FNO, seed 42, 141×141 grid, RTX A6000. Means and standard deviations over 100 synchronized repeats after 20 warm-ups, in milliseconds.
  • Width Selection: Narrowing channels does reduce cost. An input-only binary width selector (choosing between 8 and 32 channels) achieved 0.52–0.95× the latency of the dense model.
  • Coverage Shortfalls: On the new 1,000-example test set, the adaptive selectors often fell below their 90% coverage target. In contrast, static models calibrated under the same rules generally achieved higher coverage and were frequently faster.
  • Roadmap of Approaches: The authors categorized their findings into four distinct strategies, summarized in their roadmap (Table 1).
Table 1: Roadmap: the four approaches studied in this paper. Runtime findings apply only to the tested FNO and hardware.

Limitations and Future Work

  • Hardware Specificity: The latency results are highly dependent on the specific GPU (RTX A6000) and FNO implementation. Higher-resolution or 3D problems might show different scaling.
  • Distribution Shift: Conformal guarantees rely on the data being "exchangeable." The authors showed that coverage collapses entirely under severe distribution shifts (e.g., changing fluid viscosity), highlighting the need for OOD (Out-of-Distribution) detection.
  • Learned Routers: The study used a simple hand-crafted statistic (TV) for routing. Future work could explore lightweight learned scorers that might provide better predictive power for model error.

Summary updated

Abstract

Selecting less computation in a neural operator need not reduce inference time. We study this gap using conformal calibration to choose Fourier modes or channel width per input subject to a fixed error budget. A triangle-inequality envelope bounds all candidates simultaneously; a direct-score rule handles non-monotone candidate errors and, with an input-independent score, calibrates a static baseline under the same marginal guarantee. Under frozen-model and exchangeability assumptions, both rules bound the probability of issuing a prediction outside the budget and abstain when calibration is infeasible. On Burgers and Darcy flow across three training seeds, mode selection reduces active spectral support without a measurable latency reduction, because full-grid transforms and channel mixing dominate the cost. Narrowing channels does reduce Darcy forward cost, but reference-based selection must first compute the dense prediction. An input-only binary width selector avoids that pass and runs at 0.52-0.95 times dense latency; yet static widths calibrated under the same rule are often as fast or faster, and on a separately generated 1,000-example test set they generally achieve higher coverage, while the selector falls below its 90% target in more than half of the settings. Adaptive computation in neural operators should therefore be judged by achieved coverage and complete runtime against a matched static comparator.
BibTeX
@misc{ibrahim2026conformally,
  author = {Abdul Qadir Ibrahim and Martin Burger and Abel Maviano Jonathan},
  title = {{Conformally Calibrated Adaptive Computation in Neural Operators}},
  year = {2026},
  eprint = {2611.23440},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23440},
}

Loading related content...

Figure