Where Should a Neural PDE Solver Change When the Geometry Keeps Changing?

Neural operators are fast learned surrogates for physics simulations: networks that take a problem description, such as an airfoil shape and inflow conditions, and output the whole flow field in milliseconds rather than the roughly 25 minutes on 16 CPU cores that a solver needs per case in AirfRANS. In real design work, new shape families arrive one after another. An engineering team moves from one airfoil family to another, from one cavity layout to the next, and the surrogate must learn each new family quickly. It should do this without losing accuracy on the shapes it already handled, and without storing every past simulation, since each one can have hundreds of thousands of mesh cells. Every irregular-geometry operator in the literature (Transolver, GNOT, adaptive Mamba operators) is trained once on a fixed pool of shapes. None of them is studied under this sequential setting.

We propose to route all shape-specific adaptation through a small, separate pathway. A geometry encoder produces per-channel scales and shifts that are applied to the solver's early layers; this is FiLM, feature-wise linear modulation, an affine rescaling of hidden activations computed from a side input. This pathway is meta-trained so that a few gradient steps fit a new shape family. The shared physics layers, meanwhile, are protected by a tiny replay buffer. The buffer holds only a few thousand solution values per past simulation, concentrated near walls, and never full flow fields. The scientific question underneath is one the literature leaves unresolved. Injecting geometry into intermediate layers is known to fix depth-wise loss of shape information. It is not known whether that same injection point makes sequential forgetting better or worse.

Core Claim

Do Neural Operators Forget Geometry? shows that deep operators lose information about the domain shape as activations pass through repeated global mixing layers. Re-injecting the shape with FiLM repairs this: the authors report a 57.4% error reduction for Transolver on AirfRANS with early injection and 66.6% for FNO on FlowBench-LDC with late injection. They also warn that late injection in Transformer operators produces a "geometric shortcut", in which early layers go inactive and the late modulation maps coordinates straight to outputs. The continual-learning recipe from Meta-Learning Representations for Continual Learning (OML) prescribes the opposite placement: freeze deep features and adapt only late, shallow layers.

Our claim is that for sequences of geometries, plasticity belongs in early-layer FiLM generated from the geometry, not in late layers or the output head. Late plasticity invites the shortcut. Once the late layers become the only geometry-dependent part, they collapse onto the newest family, which mirrors the head collapse reported in Heads collapse, features stay. Early FiLM plasticity, combined with sparse replay that anchors the trunk, should give both fast adaptation and low forgetting at a buffer far smaller than one full simulation per past case. The claim is falsifiable. If late-layer or head-only plasticity matches early FiLM on forgetting and adaptation speed, the claim is wrong.

Research Program

The model. We start from Transolver (Transolver), which groups mesh points into learned "physics slices" and runs attention over the slices. It is chosen because it runs natively on unstructured meshes and is the backbone on which geometry injection was shown to help. A geometry encoder $e_\phi$ reads the boundary point cloud of domain $\Omega$ and outputs a code $z_\Omega$. At a chosen set of layers $\mathcal{S}$, the hidden features $h_l$ are modulated as

$$h_l \leftarrow \gamma_l(z_\Omega) \odot h_l + \beta_l(z_\Omega), \quad l \in \mathcal{S},$$

where $\gamma_l$ and $\beta_l$ are small heads that produce per-channel scale and shift. We write $\phi$ for all encoder and FiLM-head parameters, and $\theta$ for the shared trunk. The set $\mathcal{S}$ is the main experimental variable: it can be early layers, late layers, or the output head only.

Meta-training for fast adaptation. The pretraining pool is split into geometry families, and we simulate the deployment stream during pretraining. In the spirit of the look-ahead meta-update of MetaCLGraph, each meta-step takes $k$ inner gradient steps on $\phi$ alone, using a support batch from a "new" family $\mathcal{T}_t$. The outer loss is then evaluated on both the new family and a replay batch $\mathcal{B}_\mathcal{M}$ drawn from earlier families:

$$\min_{\theta,\phi}\; \mathcal{L}_{\mathcal{T}_t}^{\text{query}}(\theta, \phi'_t) + \lambda\, \mathcal{L}_{\mathcal{B}_\mathcal{M}}(\theta, \phi'_t), \qquad \phi'_t = \phi - \alpha \nabla_\phi \mathcal{L}_{\mathcal{T}_t}^{\text{supp}}(\theta,\phi),$$

where $\alpha$ is the inner learning rate and $\lambda$ weights replay. Because the inner loop touches only the small FiLM pathway, the second-order terms are confined to a few thousand parameters. This addresses the Hessian cost that makes full bilevel training impractical on 2D and 3D meshes. We also run a first-order variant as a check. The formulation differs from MMAML, which generates FiLM from a support set in stationary few-shot episodes on images and 1D functions. Here the FiLM pathway is conditioned on geometry, the objective explicitly penalizes forgetting, and the target is an operator on meshes.

Replay of sparse labels. The buffer $\mathcal{M}$ stores full geometries, which are cheap: AirfRANS airfoils are fixed by a few NACA digits plus inflow speed and angle, so their meshes can be regenerated. For supervision it stores solution values at only $K$ points per past simulation. The forward pass runs on the full regenerated mesh, and the replay loss is computed only at the stored points. This keeps the physics-slice statistics intact while cutting storage by roughly two orders of magnitude.

Points are chosen by a boundary-weighted rule that oversamples cells near the wall, where $y^+ \approx 1$, and high-curvature surface regions. The motivation comes from AirfRANS: models with low volume error still get drag wrong because they miss boundary-layer gradients. We compare this rule against uniform point selection at equal $K$. This is the operator analogue of the boundary-targeted replay in Retrospective Adversarial Replay and of coverage-based selection in MetaCLGraph.

Diagnosis. To test the mechanism rather than just the scores, we attach the layer-wise geometric probes of Xia and Aviles-Rivero after every stage of the stream. For each layer we record how well the probes reconstruct the current and past geometries. We also record how much each layer's weights moved and how active the early layers remain. The shortcut hypothesis predicts a specific pattern for late-layer FiLM. Early-layer probe accuracy should drop over the stream, and forgetting should be concentrated in the modulated late layers.

Baselines

The naive reference is sequential fine-tuning: the same Transolver with input-only geometry, trained on each family in turn, with no replay and no meta-training. This measures raw forgetting.

The strongest existing competitor is Transolver with early FiLM geometry memory, as in Do Neural Operators Forget Geometry?, trained sequentially with standard random replay. The replay uses a 50/50 current-to-replay batch ratio, following Continual Learning with Tiny Episodic Memories, and the same buffer size in bytes as our method. This is the proposal minus meta-training and minus boundary-weighted selection, so it is the most direct threat. For its static accuracy, the file reports a 57.4% error reduction on AirfRANS relative to plain Transolver. For another AirfRANS reference point, Graph U-Net reaches a lift Spearman correlation of 0.967 in AirfRANS. No reported result was found on Lacuna for any neural operator trained continually across a sequence of geometry families, so every sequential number in this study will be new, including those for baselines.

We also run elastic weight consolidation (EWC), a parameter-regularization method with no buffer, as the standard regularization competitor. We run joint training on all families at once as the upper bound on accuracy. We run an OML-style variant that freezes the trunk after pretraining and adapts only the output head, which tests the continual-learning prescription directly.

Three ablations isolate the key ingredients. The first moves the FiLM layer set $\mathcal{S}$ from early to late layers to the head only, with everything else fixed. The second removes the meta-training. The third replaces boundary-weighted point selection with uniform selection.

For fairness, all methods share the backbone size and the total number of gradient steps per stage. The meta-training cost is charged against our method's pretraining budget. All methods get the same replay byte budget and the same hyperparameter search budget, set in trials per method.

Beating the strong baseline means lower average forgetting and equal or lower final relative L2 error, with a gap larger than the seed-to-seed spread over three seeds. It also means fewer adaptation steps to reach the baseline's new-family error. Drag error should not get worse.

Why not just…?

Why not just use a strong operator with random replay? This is the most serious alternative, and it may be enough. MetaCLGraph's gap over random replay (64.73% vs 2.26% average performance on CoraFull-CL) comes from graph classification, where heads collapse. Continuous regression may be far more forgiving. The matched-budget comparison above settles it. If random replay with early FiLM matches our method at every buffer size, the meta-training and point-selection components are unnecessary, and we will report exactly that.

Why not just freeze the trunk and store one FiLM code per family? This is parameter isolation. It gives zero forgetting almost for free, because FiLM codes are tiny, and it is a real threat. It fails only if later families need changes in the trunk itself, not just re-modulation. That is plausible for moves such as higher Reynolds number, where the physics regime shifts. We include frozen-trunk FiLM codes as a baseline and measure forward transfer on the late, physics-shifted families. If frozen codes match our method there, we drop trunk replay and propose the simpler method.

Why not just retrain jointly on everything? Joint training is the accuracy ceiling. It is the right choice when every past simulation can be kept and retraining costs nothing. Our setting assumes a storage budget of a small fraction of one field per past case and bounded compute per new family. We report joint training so the remaining gap is explicit.

Why not just use EWC and skip the buffer? MetaCLGraph reports that EWC and memory-aware synapses (MAS, a related regularizer) forget badly under shifts in graph structure, and Chaudhry et al. find that tiny buffers beat regularization. Neither result is on PDE operators, so EWC stays in as a baseline rather than being dismissed.

Proposed Experiments and Decisive Checks

We build three geometry-incremental streams from benchmarks in the file. The first is Darcy flow on polygonal domains, ordered pentagon, then hexagon, then octagon, with a held-out family for adaptation tests. The second is FlowBench-LDC cavities grouped into shape families. The third is AirfRANS, split into five stages: two camber groups of NACA 4-digit airfoils, then 5-digit airfoils, then the high-Reynolds range, then the high-angle-of-attack range, using the dataset's own extrapolation ranges.

After every stage we record the relative L2 error on each past family. From these we compute average performance and average forgetting, as in the continual-learning benchmarks. On AirfRANS we also report errors in the lift and drag coefficients and the Spearman correlation for lift. For each new family we report adaptation curves: error after 1, 5, and 25 steps. We sweep the replay budget $K$ from 256 to 8192 points per simulation.

The decisive check is the placement ablation, read alongside the probes. Our claim survives only if three things hold together. Early FiLM must forget less than late or head-only plasticity. Late placement must show the early-layer collapse signature in the probes. Boundary-weighted replay must keep drag error flat at small $K$ where uniform replay does not. If late plasticity forgets equally little, or frozen-trunk FiLM codes suffice, the note's central claim is refuted, and the outcome reduces to a practical recommendation for the simpler method.