Some differential equations have a tiny parameter $\varepsilon$ in front of their highest derivative, and their solutions contain boundary layers: thin strips, often only $\varepsilon$ wide, where the solution changes almost instantly. Neural PDE solvers handle these badly. A standard Fourier neural operator (a network that learns to map an input function, such as a forcing term, to the solution function by mixing a fixed number of Fourier modes) must keep more and more modes as $\varepsilon$ shrinks, so its size grows with $1/\varepsilon$. The methods that do succeed get around this by being told what the layer looks like and where it sits, through hand-derived "corrector" functions. We propose a different route. The network outputs the solution as a quantized tensor train: the values on a dyadic grid of up to $2^{40}$ points are stored as a chain of small matrices, one per level of grid refinement. It is trained without data by minimizing a preconditioned PDE residual that is computed exactly in this compressed format. Earlier numerical analysis proved that such tensor trains represent boundary-layer solutions with ranks that grow only polylogarithmically in the accuracy and do not depend on $\varepsilon$. If a network can learn to produce them, its size should stay flat as $\varepsilon \to 0$, and it should need no advance knowledge of where the layers are or what shape they have.
In short, we want to move model size for boundary-layer problems from growing like $1/\varepsilon$ to staying roughly constant in $\varepsilon$. We do this by changing what the network outputs rather than by compressing its weights. The headline measurement is a curve: for each method, the number of parameters needed to reach a fixed error as $\varepsilon$ goes from $10^{-2}$ to $10^{-8}$.
Core Claim
The claim has two parts. First, a small encoder can map the problem data (forcing $f$, coefficients $b, c$, and $\log\varepsilon$) to the cores of a rank-$r$ quantized tensor train (QTT) on a virtual grid with $2^L$ points. One output head, conditioned on the level index $\ell \in \{1,\dots,L\}$, produces the core for every level. Parameter count is then independent of $L$, and therefore of how finely the grid resolves $\varepsilon$. Second, training can be done without precomputed solutions. The loss is
$$\mathcal{L}(\theta) = \mathbb{E}_{(f,b,c,\varepsilon)} \big\| P_\varepsilon \big( A_\varepsilon\, u_\theta - f \big) \big\|_2^2 .$$
Here $u_\theta$ is the network's QTT output, $A_\varepsilon$ is the finite-difference operator on the $2^L$ grid, and $P_\varepsilon$ is a multilevel (BPX-type) preconditioner. The discrete Laplacian and the preconditioner both have low-rank QTT forms, so the loss can be computed exactly by tensor contractions at a cost polynomial in $L$ and $r$. No collocation points are sampled. The preconditioner addresses the failure identified by Chen et al., who show that ordinary residual training stalls as $\varepsilon$ shrinks because of an eigenvalue bottleneck in the discrete operator. Our view is that this is a conditioning problem, and conditioning is what $P_\varepsilon$ is built to fix.
Where This Sits
The most important prior result is Marcati, Rakhuba and Ulander. They solve single 1D reaction-diffusion problems in QTT format on grids up to $2^{50}$ points. They prove ranks independent of the perturbation parameter and reach stability down to $10^{-16}$ using a modified BPX preconditioner. We reuse their representation and preconditioner. Their work has three limits that we address: each instance is solved from scratch, only constant-coefficient reaction-diffusion is treated, and nothing is learned across instances. We amortize the solve across a family of inputs and extend the setting to variable-coefficient convection-diffusion.
The asymptotics-informed operators play two roles: they motivate the proposal, and they are its main competitors. ComFNO adds a second FNO plus a dense layer that fits an exponential layer profile in the zoomed coordinate $(x_0-x)/\varepsilon$. It improves on vanilla FNO, but it increases parameter count and assumes the layer location and its exponential form. eFEONet uses a small CNN to predict coefficients for finite-element basis functions enriched with known corrector functions. It trains data-free on the weak form and reaches roughly $10^{-5}$ relative $L^2$ error in 1D. SL-PINNs strip out the layer analytically, so a 2-layer, 32-neuron network suffices down to $\varepsilon=10^{-6}$. These three papers show that the network can be tiny once the layer is taken care of. All three take care of it by hand. We keep the small network and replace the hand-derived corrector with a representation that finds layers wherever the residual puts them.
On the model-size side, Cao's Galerkin Transformer reduces the cost of attention through structure, not pruning. The block-circulant compression of Hayashi and Shimbo reduces an $O(d^2)$ matrix to $O(d)$. Both compress weights. We instead compress the output, which is where the $1/\varepsilon$ cost actually comes from. Weight compression applied to an FNO is therefore an ablation we run, not the method.
Research Program
We build the operator in 1D first. The encoder is a small FNO or CNN that reads $f, b, c$ on a coarse grid of 256 points. The output head takes the encoder's latent vector and a learned embedding of level $\ell$, and produces a $2\times r\times r$ core. We apply a rounding step (a truncated SVD sweep) only at evaluation time, and train through the unrounded cores.
We then run four studies. The first is scaling. For each $\varepsilon \in \{10^{-2},\dots,10^{-8}\}$ and each method, we find the smallest parameter count that reaches $10^{-3}$ relative $L^2$ error. We expect QTT operators to stay flat and FNO-family methods to grow. The second tests unknown layers. These are turning-point problems where $b(x)$ changes sign at an input-dependent location, producing interior layers whose position varies across the dataset, plus problems whose layers are exponential in some samples and parabolic in others. The third tests the training mechanism by comparing preconditioned and unpreconditioned residual losses, and by comparing a curriculum over $\varepsilon$ (in the style of the homotopy approach) with direct training at the target $\varepsilon$. The fourth extends to 2D convection-diffusion on the unit square, using interleaved-bit QTT for the tensor-product grid. This covers exponential and corner layers, following the setting in Gie et al..
Baselines
The naive reference is a vanilla FNO trained on solutions computed on fine Shishkin meshes, swept over mode count and width. It shows how parameter count must grow with $1/\varepsilon$ when the architecture is not changed. On 2D elliptic problems, ComFNO reports vanilla FNO at $L^\infty = 1.67\times10^{-2}$ against ComFNO at $3.0\times10^{-3}$. We reproduce both on the same generated data before comparing anything else.
The strongest existing method is eFEONet. It is data-free like ours, and Lee et al. report relative $L^2$ errors as low as $10^{-5}$ in 1D, two orders of magnitude better than ComFNO. We run it in two regimes. In the first it receives the correct correctors; there we expect it to match or beat us, and we count parity within a factor of 3 in error at no more parameters as success. In the second the correctors are wrong or missing, as in turning-point interior layers or mixed layer types. Beating it there means at least an order of magnitude lower error at matched parameter count.
ComFNO is run at its published configuration and at matched parameter count. We also count its parameters, since neither ComFNO nor eFEONet reports a parameter count; no reported result was found on Lacuna for model size on these benchmarks.
The per-instance QTT solver of Marcati et al. serves as the accuracy ceiling and the wall-clock reference. It sets the error our amortized model can at best approach, and the time per query it must beat to be worth using.
Two ablations isolate what is new. The first unties the output heads, with one head per level, to test whether sharing weights across scales is what keeps size flat. The second replaces the QTT output with a dense grid output while keeping the same encoder and the same preconditioned loss, to test whether the output representation is doing the work. All methods receive the same input distributions, the same GPU-hour budget, and the same hyperparameter search of 30 trials. Error is measured on held-out inputs against reference solutions from fine Shishkin meshes.
Why not just…?
Why not just give the FNO a Shishkin-mesh or stretched-coordinate input? When layers sit at known boundaries, this is a real threat. It is essentially the trick ComFNO already uses, the coordinate zoom $(x_0-x)/\varepsilon$, applied to the whole network. It fails when layer positions depend on the input, because a single fixed mesh cannot cluster points at a moving turning point. We include a Shishkin-input FNO as an additional baseline. If it matches us on the moving-layer benchmark, the representation is not needed and we will say so.
Why not just use eFEONet with better correctors? When the asymptotics are known, it probably wins, and we do not claim otherwise. The case for our approach rests entirely on problems where deriving correctors is impractical, which Li et al. and Gie et al. both leave open. The misspecified-corrector experiment tests exactly this.
Why not just run the QTT solver per instance? If a preconditioned QTT solve takes milliseconds, amortizing it may not pay off. We report the break-even number of queries, including training cost. The learned model has to justify itself either on many-query throughput or on problem classes, such as variable convection, where the per-instance solver's rank guarantees have not been established. If neither holds, the negative result is still informative.
Why not just train a large FNO and prune it or make its weights low-rank? The file notes that oversized models on stiff problems often never converge to the right solution. Even when they do, pruning cannot remove the $1/\varepsilon$ mode count that the output grid demands. The low-rank-weight FNO ablation measures this directly on the scaling curve.
Decisive Checks
The idea succeeds if three things hold. The parameters-to-$10^{-3}$ curve for the QTT operator grows by less than a factor of 2 between $\varepsilon=10^{-2}$ and $10^{-8}$, while every FNO variant grows by more than 10×. On moving interior layers, it beats misspecified eFEONet by an order of magnitude. And dropping the preconditioner makes training at small $\varepsilon$ fail, which would confirm that conditioning, not capacity, is the bottleneck. If the untied-head ablation performs as well as the shared head, the model-size argument weakens and we would present the result as a representation result only. If the 2D results stay flat in $\varepsilon$, the method is ready for curved domains, which would require mapping the domain to a tensor-product reference.
References
- Learn Singularly Perturbed Solutions via Homotopy Dynamics
- Low-rank tensor approximation of singularly perturbed boundary value problems in one dimension
- Component Fourier Neural Operator for Singularly Perturbed Differential Equations
- Data-free Asymptotics-Informed Operator Networks for Singularly Perturbed PDEs
- Semi-analytic PINN methods for boundary layer problems in a rectangular domain
- Choose a Transformer: Fourier or Galerkin
- A Non-commutative Bilinear Model for Answering Path Queries in Knowledge Graphs