Keeping the Buffer Zone Honest: Frame-Projected Fourier Extension for Neural Operators

Neural operators are networks that learn to map one function to another, for example a PDE's coefficients to its solution. Many of the fastest ones, such as the Fourier Neural Operator, work in frequency space and therefore implicitly assume the domain is periodic. Real problems have walls, inflows and lids. A standard fix, Fourier extension (also called Fourier continuation), embeds the physical interval into a larger periodic box. Inside the padding region, or "buffer zone", the function may do whatever it needs to wrap around smoothly. This works well when one function is being approximated. Inside a learned operator, though, it creates a hidden problem. The buffer zone is never supervised by data, yet every spectral layer is a convolution on the extended box, so whatever the network writes into the buffer is mixed back into the physical domain at the next layer. The buffer thus becomes an unregularized side channel whose contents depend on arbitrary choices: which extension was used, at what resolution, and with what weights.

This note proposes treating each layer of a Fourier-extension neural operator as a discretized operator in the sense of frame theory. A frame is a redundant, overcomplete spanning set, used here instead of an orthonormal basis. Concretely, after every learned spectral map, the network evaluates the result only on the physical domain and re-fits it with a truncated-SVD Fourier extension. This removes the components that live only in the buffer. The claim is that this single projection makes the learned operator a well-defined map on functions over the physical domain. Its predictions should then be invariant to the choice of extension, and it should generalize better across resolutions and out-of-distribution inputs than architectures that extend once and then let the buffer drift.

Core Claim

On the Numerical Stability of Fourier Extensions shows two things. First, complex exponentials on an extended box, restricted to the physical interval, form a redundant frame. Second, the singular vectors tied to its tiny singular values live almost entirely in the buffer and vanish on the physical domain, so least-squares fits are stable despite severe ill-conditioning. That result concerns approximating a single function. Its implication for learned operators has not been examined: the same near-null directions are exactly the directions a trained network can fill freely without the loss ever noticing.

Fourier Continuation for Exact Derivative Computation in Physics-Informed Neural Operators (FC-PINO) extends the input once, runs an FNO on the extended box, and restricts at the end. It reports PDE residuals near $10^{-12}$ on 1D Burgers, where standard physics-informed operators stall between $10^{-3}$ and $10^{-5}$. FC-PINO does nothing to stop buffer content from accumulating across its internal layers. We hypothesize that this is harmless for smooth single-regime problems but becomes a source of instability and poor transfer when inputs are less smooth or shift in distribution.

The frame-theoretic fix comes from operator discretization. Matrix Representation of Operators Using Frames and U-cross Gram matrices and their invertibility represent an operator $U$ through the matrix

$$G_{U,\Phi,\Psi} = T_\Phi^* \, U \, T_\Psi,$$

where $T_\Psi$ synthesizes a function from coefficients in frame $\Psi$, and $T_\Phi^*$ analyzes the output against frame $\Phi$, here taking inner products on the physical domain only. The second paper proves that exact invertibility requires Riesz sequences, which Fourier extension frames are not. For true frames, it shows the Moore–Penrose pseudo-inverse is again a $U$-cross Gram matrix built from a specific dual frame. We do not need exact inversion. We need the finite-precision, truncated analogue. Our layer replaces the FNO update $c \mapsto \sigma(Wc)$ on extended-box Fourier coefficients $c$ with

$$c \mapsto P_\varepsilon \, R \, \sigma(W c), \qquad P_\varepsilon = V_\varepsilon \Sigma_\varepsilon^{-1} U_\varepsilon^{*}.$$

Here $W$ is the learned spectral multiplier and $\sigma$ the nonlinearity. $R$ evaluates the extended-box function at oversampled physical-domain points. $P_\varepsilon$ is the truncated-SVD Fourier-extension fit that keeps only singular values above a threshold $\varepsilon$. The composite map $P_\varepsilon R$ is a projector onto the physically visible part of coefficient space, and it annihilates buffer-only modes.

Because the state stays in coefficient space between layers, this connects naturally to Transform Once: Efficient Operator Learning in Frequency Domain (T1). T1 stays in frequency space throughout for 3x to 10x speedups but names non-periodic boundaries as an unsolved limitation. Our projection is what allows coefficient-space propagation to remain meaningful on a non-periodic domain.

Research Program

The work has three parts.

Construction. We build the projected layer in 1D, 2D and 3D on tensor-product boxes. The extended period is fixed at twice the domain, following Parameter selection and numerical approximation properties of Fourier extensions from fixed data, which shows that the extension length barely matters once the target condition number is fixed. This permits power-of-two FFTs. $P_\varepsilon$ is precomputed once per resolution, applied per dimension, and kept fixed during training. Doing so costs an SVD of a modest matrix up front, after which applying it costs a dense matrix product. One open engineering question is whether a fast structured solver is needed at 3D resolutions; we will measure this rather than assume it.

Analysis. We prove a truncated counterpart to the pseudo-inverse theorem of U-cross Gram matrices and their invertibility. The goal is to show that each projected layer, viewed as a map from functions on the physical domain to functions on the physical domain, has operator norm bounded by the learned multiplier's norm times a constant depending only on $\varepsilon$ and the frame bounds. The unprojected FC layer has no such bound, because buffer modes can be amplified without limit.

We then carry this stability into a sample-complexity statement. We will adapt the criterion of On the Stability and Accuracy of Least Squares Approximations, under which a Gram matrix is close to identity when the basis-to-sample ratio stays below $n/(c\log n)$. The adaptation replaces the nominal number of Fourier modes with the number of singular values retained above $\varepsilon$, which we expect to be what governs generalization. Two results set honest limits on what can be claimed. A Stability Barrier for Reconstructions from Fourier Samples caps stable convergence for non-periodic analytic functions at root-exponential rates. Minimax rates of order $n^{-2s/(2s+d)}$ for fractional Sobolev targets, from Minimax Optimal Nonsmooth Nonparametric Regression via Fractional Laplacian Eigenmaps, give the reference curve that error-versus-training-set-size plots should be read against when solutions are rough.

Diagnostics. We introduce two measurements:

  1. Buffer energy is the fraction of a hidden state's norm living in the near-null singular directions.
  2. Extension sensitivity is the change in the physical-domain prediction when the input is extended with FC-Gram rather than FC-Legendre at test time, the two extension schemes used by FC-PINO.

A well-defined operator on the physical domain should have near-zero extension sensitivity. Our prediction is that unprojected FC operators do not.

A secondary, optional thread uses localized Fourier extensions on overlapping subdomains, following A Local Fourier Extension Method for Function Approximation, as the subspaces of a fusion frame (a frame made of weighted subspaces, introduced in Modeling sensor networks with fusion frames). This would replace the heuristic POD partitions of Transfer Operator Learning with Fusion Frame with subspaces whose stability is governed by the perturbation bounds in The Invertibility of U-Fusion Cross Gram Matrices of Operators. We pursue it only if the single-domain projection pays off.

Baselines

Naive reference. The do-nothing reference is a standard FNO from Fourier Neural Operator for Parametric Partial Differential Equations, applied directly to non-periodic data with its spatial skip path and zero padding. The FNO paper argues the skip path handles non-periodicity. FC-PINO reports that this kind of baseline stalls at residuals of $10^{-3}$ to $10^{-5}$ on 1D Burgers and fails to satisfy the equations on 3D lid-driven cavity flow. It tells us how much any extension buys.

Strongest competitor. The strongest existing method is FC-PINO with both FC-Gram and FC-Legendre extensions. It is the only published method that already combines Fourier extension with a neural operator on non-periodic domains, so it is the direct target. Its reported figure is a PDE residual of $10^{-12}$ on 1D Burgers. For 2D viscous Burgers and 3D cavity flow, no reported numeric result was found on Lacuna, only qualitative convergence. We will retrain it from the authors' description on identical data, with identical parameter counts, an identical hyperparameter search budget, and identical GPU-hours.

On smooth single-regime tasks we do not expect to beat its residual and will not claim to. The proposal counts as beating FC-PINO if three conditions hold together:

  1. Physical-domain relative test error is at least comparable in-distribution.
  2. Error is lower under resolution change and input-roughness shift.
  3. Extension sensitivity is lower by an order of magnitude.

Coefficient-space speed reference. T1 serves as the coefficient-space speed reference, run with its variance-preserving initialization. It reports over 20% lower error than FNO on 2D Navier–Stokes, a periodic task. On non-periodic tasks we expect it to suffer.

Data-driven subspace reference. For Darcy flow, FF-POD-DeepONet is the data-driven subspace reference. Its reported source-domain MSE is 0.0589 against FNO's 0.0671, and its 20-sample transfer MSE is 0.2074 against POD-DeepONet's 0.3025. We use these as reported, with the caveat that their exact data split must be matched before any comparison is called a win.

Ablations. The key ablation removes the projection $P_\varepsilon R$ while keeping everything else, including coefficient-space propagation and the extension. A second ablation varies $\varepsilon$ to trace the trade-off between keeping more modes and admitting more buffer drift.

Why not just…?

Why not just use FC-PINO as it is? It may be enough. If our diagnostics show low buffer energy and low extension sensitivity in FC-PINO across all our shift conditions, the hypothesis is wrong, and we will report that. The reason to expect otherwise is structural. Nothing in FC-PINO's internal layers constrains buffer content, and the frame analysis in On the Numerical Stability of Fourier Extensions says these directions are invisible to any loss computed on the physical domain. The extension-swap test settles the question directly.

Why not just add a penalty on buffer-zone energy to the loss? This is the most serious threat, because it is a one-line change. A soft penalty discourages drift on training inputs but does not guarantee it on shifted inputs, and it adds a weight that needs tuning. The hard projection costs a fixed matrix product and needs no tuning beyond $\varepsilon$. We will run the penalty variant with its weight tuned on the same budget. If it matches the projection under shift, we will recommend the penalty and keep only the theory.

Why not just use a Chebyshev or Legendre basis, which is designed for non-periodic intervals? Polynomial bases require clustered grids or suffer instability on equispaced data. On the Stability and Accuracy of Least Squares Approximations shows the stable number of polynomials on uniform samples grows only like the square root of the sample count. Fourier extension keeps equispaced FFT grids, which most PDE datasets already use. We will nonetheless include a Chebyshev-coefficient operator, with data interpolated to Chebyshev nodes, so that this argument is tested rather than asserted.

Why not just apply Tikhonov regularization instead of truncated SVD? Tikhonov shrinks the near-null directions rather than removing them, so some buffer content survives every layer and can compound with depth. We will include it as a variant. We expect it to behave between the unprojected and projected models, with the gap growing with network depth.

Proposed Experiments

Benchmarks. We use the non-periodic benchmarks already used by FC-PINO: 1D Burgers, 2D viscous Burgers, and 3D lid-driven cavity flow. We add the PDEBench Darcy and reaction–diffusion tasks with non-periodic boundaries from PDEBench: An Extensive Benchmark for Scientific Machine Learning, reporting its boundary error (bRMSE) and Fourier-spectrum error (fRMSE) alongside relative L2 error. No reported per-model bRMSE was found on Lacuna, so all such numbers will be produced by our own runs. Darcy flow with permeability shifts, following Transfer Operator Learning with Fusion Frame, serves as the transfer test.

Decisive checks. There are four:

  1. Extension swap. Train with FC-Gram, test with FC-Legendre, and the reverse. The projected model should change its physical-domain prediction by roughly the extension's own approximation error. FC-PINO, we predict, changes far more.
  2. Resolution shift. Train at one resolution, test at twice and half that resolution.
  3. Roughness sweep. Generate Burgers and Darcy inputs with controlled fractional Sobolev smoothness $s$ from 0.3 to 2. Plot error against training-set size for each $s$, and compare slopes with the $n^{-2s/(2s+d)}$ reference. The aim is to see whether projection preserves the expected rate as smoothness drops while the unprojected model departs from it.
  4. Buffer-energy tracking. Track buffer energy layer by layer during training for all variants, to confirm that the mechanism, not just the outcome, differs.

Falsification. The proposal fails if the unprojected ablation matches the projected model on all four checks. That would mean buffer drift is not a practical problem, and the contribution would shrink to the stability bound.