Neural operators learn to map one function to another, for example from a rock permeability field to the pressure it produces. The most widely used one, the Fourier Neural Operator, does its core computation in a global Fourier basis, which assumes the solution is periodic. Real problems have walls, inflow boundaries and irregular shapes. Fourier approximation of a non-periodic function oscillates and converges slowly near the boundary (the Gibbs phenomenon), and the standard fix of zero-padding the grid is an engineering habit with no analysis behind it.
Applied harmonic analysis has a principled answer, and it has never been tested where it matters. Fourier extensions embed the domain in a larger periodic box and represent functions with a redundant Fourier frame, a spanning set with more elements than strictly needed. They converge geometrically down to machine precision, even though their coefficient systems are badly conditioned. A separate body of operator theory shows how to write any linear operator as a block matrix over a fusion frame, a weighted collection of overlapping subspaces. That theory proves exact invertibility only in the non-redundant (Riesz basis) case, and it contains no numerical experiments at all.
We propose to build a neural operator layer whose subspaces are overlapping spatial patches, each carrying its own small Fourier-extension frame. The operator is learned as the block representation over these patches, and we test whether redundancy, which theory treats as an obstacle to inversion, is in practice what makes such operators accurate at boundaries and robust under distribution shift.
Core Claim
Our claim is that a neural operator layer built on a deliberately redundant patch fusion frame, reconstructed through a truncated (regularized) dual, will (i) cut boundary error relative to a padded-FFT Fourier Neural Operator on non-periodic and irregular domains, and (ii) transfer better under physical-parameter shift than a non-overlapping, Riesz-basis version of the same layer. We also claim that the measured fusion frame bound ratio of the chosen patch configuration predicts that transfer gain. The second and third parts are what is new. They turn an open tension in the theory into a falsifiable empirical statement. On one side, the invertibility results require Riesz bases, i.e. zero redundancy. On the other, the compressed sensing and transfer results credit redundancy for robustness. The third part also gives a criterion for choosing subspaces automatically, which the closest empirical work leaves open.
Where This Sits
The operator-theoretic foundation comes from Representation of operators using fusion frames. That paper represents an operator $O$ by blocks $w_j v_i P_{W_j} O P_{V_i}$ between subspaces, derives explicit pseudo-inverses, and observes that overlap-add convolution is a diagonal instance of this structure. U-cross Gram matrices and their invertibility and The Invertibility of U-Fusion Cross Gram Matrices of Operators prove that such representations are invertible exactly when the frames are Riesz bases, and they give perturbation thresholds. We reuse their block structure and canonical duals as the method. Their Riesz-only invertibility result is a constraint we deliberately step outside, because none of the three papers reports a single floating-point computation.
The finite-precision justification comes from A Stability Barrier for Reconstructions from Fourier Samples. That paper proves that stable reconstruction of non-periodic analytic functions from Fourier data converges at best root-exponentially. It also shows that redundant Fourier extension frames escape this limit in practice, converging geometrically to a tolerance despite ill-conditioning. Its finding motivates our layer, but it was established for one-dimensional function approximation, not for learned operators. The same paper notes that irregular geometries produce ill-conditioned bases, and we treat that as the regime where frames should matter most. On the Stability and Accuracy of Least Squares Approximations supplies the condition that the number of grid points per patch must scale with the patch basis's envelope bound. We use it to size the patches.
The subspace view comes from Compressed sensing for fusion frames and Sparse Recovery from Combined Fusion Frame Measurements. These papers recover signals that are sparse at the subspace level by mixed $\ell_1/\ell_2$ minimization, where a penalty on each subspace's total energy drives whole subspaces to zero. They prove that recovery gets more robust as subspace dimension grows. We borrow the group penalty, applied to learned block couplings rather than to signals, to prune candidate patches. Their noiseless, exactly sparse guarantees are motivation rather than prediction for our noisy learned setting.
The closest existing work is Transfer Operator Learning with Fusion Frame. Its FF-POD-DeepONet partitions the input into hand-designed redundant subspaces, runs parallel branches, compresses each with proper orthogonal decomposition, and fuses the outputs. It reports gains on Darcy flow source error and on 20-sample transfer. We differ in three ways. Our subspaces are spatial patches with Fourier-extension bases, not learned branch inputs. The layer is literally the block operator representation with a computed dual. We measure frame bounds and conditioning and test whether they predict transfer, which that paper explicitly does not connect to fusion-frame theory. The Fourier Neural Operator for Parametric Partial Differential Equations is the base we modify. It parameterizes the kernel as a learned multiplier on truncated FFT modes, uses one global orthonormal basis, and needs a uniform rectangular grid.
Research Program
The layer works as follows. Cover the domain $\Omega$ with overlapping patches. On each patch $j$, let $W_j$ be the span of a Fourier-extension frame: Fourier modes periodic on a box enlarged by a factor $T>1$, restricted to the patch. A coarse global subspace is added so that information can travel long distances. The layer acts on a hidden function $v$ as
$$\mathcal{K}_\theta v = \sum_{j}\sum_{i \in \mathcal{N}(j)} w_j w_i\, \widetilde{P}_{W_j}\, R_{ij}^\theta\, \widetilde{P}_{W_i} v ,$$
where $w_j>0$ are patch weights, $\mathcal{N}(j)$ is patch $j$ plus its neighbours and the coarse subspace, $R_{ij}^\theta$ is a small learned matrix acting on frame coefficients, and $\widetilde{P}_{W_i}$ maps grid values to patch coefficients through a truncated-SVD least-squares solve at tolerance $\epsilon$. This truncated solve is the regularized dual that Fourier extensions use. It is a fixed matrix precomputed once per patch geometry, so the forward cost stays at a few small dense multiplies per patch.
Redundancy enters at two levels: overlap between patches, and oversampling within each patch frame. Both are controlled by the fusion frame bounds $C \le D$ in
$$C\|f\|^2 \le \sum_j w_j^2 \|P_{W_j} f\|^2 \le D\|f\|^2 ,$$
which we compute numerically for every configuration from the extreme eigenvalues of the assembled fusion frame operator $\sum_j w_j^2 P_{W_j}$.
The analysis is empirical but tied to theory. For each trained model we record five quantities: the ratio $D/C$, the smallest retained singular value of the assembled block system, the perturbation margin from The Invertibility of U-Fusion Cross Gram Matrices of Operators evaluated on the learned blocks, error versus number of modes per patch, and boundary versus interior error. For automatic subspace selection, we start from an oversized dictionary of candidate patches at several scales. During training we add a penalty $\lambda\sum_j \|R_{j\cdot}^\theta\|_F$ on each patch's block row, following the fusion-frame $\ell_1/\ell_2$ objective, and retrain on the surviving patches.
The benchmarks are those in the file. We use 2D Darcy flow with zero Dirichlet boundaries under the source-and-shift protocol of Transfer Operator Learning with Fusion Frame (shifted Gaussian random field parameters and forcing, 20 target samples). We use 1D Burgers and 2D incompressible Navier–Stokes and Darcy from PDEBench: An Extensive Benchmark for Scientific Machine Learning, whose boundary RMSE and frequency-spectrum RMSE metrics are exactly the quantities our claim concerns. We also construct irregular-domain Darcy variants (L-shaped and perforated domains, generated with a standard finite-element solver on the same random-field inputs), because the boundary argument should matter most there.
Baselines
The naive reference point is the Fourier Neural Operator as published in Fourier Neural Operator for Parametric Partial Differential Equations, with its usual zero-padding for non-periodic problems. On irregular domains it is run with masking inside the rectangular bounding box. This is the default practitioners use, and padding is itself a crude, unanalysed Fourier extension, so it is the most direct competitor on the boundary question. We sweep the padding width as a hyperparameter so that the comparison is against the best padded FNO, not a strawman. For the source error, Transfer Operator Learning with Fusion Frame reports a Darcy MSE of 0.0671 for FNO.
The strongest existing method on the transfer question is FF-POD-DeepONet from the same paper. It reports Darcy source MSE 0.0589, against 0.0671 for FNO and 0.0803 for DeepONet. With 20 target samples it reports transfer MSE 0.2074, against 0.3025 for POD-DeepONet. We re-run it ourselves from its described architecture on identical splits. The citation is for reference only, because their data generation may differ from ours. For PDEBench Darcy, Burgers and Navier–Stokes, the file transcribes only that FNO outperformed U-Nets and physics-informed networks. No reported result was found on Lacuna for PDEBench Darcy MSE or boundary RMSE, and none for FF-POD-DeepONet on Elasticity or Burgers. Those baselines will therefore be re-run, not quoted.
The key ablation removes redundancy while keeping everything else fixed. It uses the same patch layer with non-overlapping patches and critically sampled (non-extended) local bases, which makes the system a fusion Riesz basis where the published invertibility theorems apply exactly. A second ablation keeps overlap but replaces the truncated dual with the exact pseudo-inverse, to isolate the role of the tolerance. All models get matched parameter counts within 10%, the same training data, the same optimizer, the same number of hyperparameter trials (we propose 30 per method by random search), and equal GPU-hour budgets. We report means over five seeds.
The bars for beating each baseline are as follows. Against padded FNO, we need lower boundary RMSE at equal or better interior error on PDEBench Darcy, and lower relative L2 on the irregular-domain variants. Against FF-POD-DeepONet, we need lower 20-sample transfer MSE on our re-run Darcy shift protocol. Against the Riesz ablation, we need a transfer gain larger than seed variance with no loss in source error.
Why not just…?
Why not just pad the FFT more aggressively? Padding is the real threat on rectangular grids, and we expect a well-tuned padded FNO to come close to our layer on PDEBench Darcy interior error. The distinction appears in two places. First, error versus number of modes: A Stability Barrier for Reconstructions from Fourier Samples predicts that a smooth frame extension reaches tolerance geometrically, while padding with zeros introduces an artificial edge jump that the network must learn to undo. Second, irregular domains, where a bounding-box FFT has no principled way to handle the internal boundary. If padded FNO matches us on both, the boundary half of the claim is wrong, and we will report it.
Why not just tune FF-POD-DeepONet with more or better branches? Its gains are real, but its subspaces are chosen by hand, and the authors list automated subspace selection as unsolved. Adding branches does not tell anyone which configuration will transfer. Our test is whether $D/C$ and the retained singular value, computed before training on the target domain, rank configurations by transfer error. If that ranking is no better than chance across roughly 40 random patch configurations, the theory-to-practice link fails regardless of which model wins on average.
Why not just use a non-redundant decomposition, where the invertibility theory actually holds? This is the Riesz ablation, and it settles the central tension directly. The theorems in U-cross Gram matrices and their invertibility favour it on conditioning. The redundancy results in Sparse Recovery from Combined Fusion Frame Measurements and the transfer gains in Transfer Operator Learning with Fusion Frame favour overlap on robustness. If the Riesz version transfers as well, the redundancy claim is refuted.
Why not just call this overlapping domain decomposition? Structurally it is close, and we say so. The contribution is not the patch structure. It is the measurement of whether frame quantities predict learned-operator behaviour, which no paper in this literature has done.
Proposed Experiments
The first decisive check concerns accuracy at boundaries. On PDEBench Darcy and our L-shaped and perforated variants, we compare boundary RMSE, interior relative L2 and frequency-spectrum RMSE against padded FNO across mode budgets of 8 to 32 per dimension. We plot error against modes on a log scale. The claim predicts faster-than-algebraic decay to a plateau set by $\epsilon$ for our layer, and a slower or earlier-saturating curve for padded FNO.
The second check concerns redundancy and transfer. On the Darcy shift protocol with 20 target samples, we compare overlapping, Riesz, and exact-pseudo-inverse versions against re-run FF-POD-DeepONet. We vary overlap fraction from 0 to 50% and extension factor $T$ from 1 to 2 to trace transfer error as a function of $D/C$.
The third check concerns predictive frame quantities. Across about 40 random patch configurations, we compute $D/C$ and the retained singular value before fine-tuning, and report their Spearman rank correlation with target-domain error. We then test whether group-penalty pruning from an oversized dictionary lands on configurations in the best quartile.
The fourth check concerns computational cost, since the file documents the absence of FLOP or wall-clock reporting for frame discretizations. We report FLOPs, wall-clock inference and training memory against FNO's FFT path on 64×64 to 256×256 grids, and the condition numbers of the precomputed patch duals. This tests whether the layer is a usable operator or only an accurate one.
A positive outcome on the first three checks would give the fusion-frame operator theory its first empirical footing and a working subspace selection rule. A negative outcome on the second or third would be equally informative. It would show that, for learned operators, the Riesz-basis view in the invertibility theorems is the right guide after all.
References
- Representation of operators using fusion frames
- U-cross Gram matrices and their invertibility
- The Invertibility of U-Fusion Cross Gram Matrices of Operators
- A Stability Barrier for Reconstructions from Fourier Samples
- On the stability and accuracy of least squares approximations
- Compressed sensing for fusion frames
- Sparse Recovery From Combined Fusion Frame Measurements
- Transfer Operator Learning with Fusion Frame
- Fourier Neural Operator for Parametric Partial Differential Equations
- PDEBench: An Extensive Benchmark for Scientific Machine Learning