Some of the most interpretable models of dynamical systems work in two parts. First, a learned invertible warp reshapes the messy observed data. Second, a small, simple system runs inside the warped space, for example a handful of linear oscillating or decaying modes whose frequencies and decay rates a hydrologist or biologist can read directly. These models tend to look excellent on simulated data and disappointing on real data. The clearest example is streamflow: the same model beats an LSTM on noisy synthetic data but loses badly to a tuned LSTM on a real Utah creek. Because simulated data are cheap, the natural response is to pretrain on simulations and then adapt on scarce real data. Doing that naively, however, means retraining everything. That erases the readable dynamics we cared about and hides what the simulator got wrong.
We propose split transfer. Pretrain the two-part model on simulated trajectories. Then adapt it to real data under three separate regimes: only the warp moves, only the simple latent dynamics move, or both move with a penalty for leaving the simulated values. The regime that best explains the real data tells you where the simulator is wrong. If adapting the warp alone recovers real-data accuracy, the gap lies in how the system is observed and distorted (sensors, noise, nonlinear gauging), and the simulator's frequencies and decay rates can be kept and read as physics. If the latent dynamics must move, the simulator's physics is off, and by how much can be read in the same units. The practical claim is that this split lets synthetic data substitute for a larger share of the real data than full fine-tuning or black-box pretraining, while keeping the explanation intact.
Core Claim
The model pairs a diffeomorphism with a structured latent operator. A diffeomorphism is a smooth invertible map with a smooth inverse, here built from coupling layers. The model is $z_t = \phi(x_t)$, $z_{t+1} = K z_t$, where $x_t$ is the observed state, possibly delay-embedded and augmented with a memory variable as in LDDMD. Here $\phi$ is the invertible warp, and $K$ is a block-diagonal matrix whose 2×2 blocks each encode one frequency $\omega_j$ and growth rate $\rho_j$. The warp carries geometry, and $K$ carries the interpretable physics. The claim has two parts. The first is a measurement: adapting $\phi$ alone versus $K$ alone after synthetic pretraining gives a faithful, component-level attribution of the synthetic-to-real gap. The second is a practical result: when the attribution says "geometry," freezing $K$ gives a better synthetic-for-real exchange rate than any regime that adapts everything, and it keeps the modal parameters authentic, meaning parameters refit from the model's own simulations match the parameters fit to real data.
The anchored joint regime minimizes
$$\mathcal{L} = \sum_t \lVert x_{t+1} - \phi^{-1}(K\phi(x_t)) \rVert^2 + \lambda \sum_j \big[(\omega_j - \omega_j^{\text{sim}})^2 + (\rho_j - \rho_j^{\text{sim}})^2\big],$$
where $\omega_j^{\text{sim}}, \rho_j^{\text{sim}}$ are the values learned on synthetic data. The penalty weight $\lambda$ traces a path between "trust the simulator's physics" ($\lambda \to \infty$, warp-only) and "ignore it" ($\lambda = 0$). The accuracy lost as $\lambda$ grows is our quantitative version of the accuracy–interpretability trade-off.
Relation to Prior Work
The mechanism comes from three papers.
- Learning Efficient and Robust Ordinary Differential Equations via Invertible Neural Networks shows that an invertible network can absorb spatial complexity while a closed-form linear ODE handles time.
- Latent Diffeomorphic Dynamic Mode Decomposition (LDDMD) adds a memory variable and a block-diagonal operator, which makes the frequencies readable.
- Learning Dynamics Models with Stable Invariant Sets shows that topology fixed in the latent space survives the warp.
We reuse this architecture unchanged. None of these papers studies what happens when one part is trained on simulations and the other on reality.
LDDMD also supplies the motivating evidence. It reports NSE 0.94 against an LSTM's 0.80 on noisy synthetic streamflow, and it recovers the synthetic frequency almost exactly ($\omega = 0.0097$ versus a true value of $0.0099$). On real Muddy Creek streamflow it falls to NSE 0.46 against a tuned LSTM's 0.89, and its authors attribute this to high-frequency noise. Here NSE is Nash–Sutcliffe efficiency, a hydrology score where 1 is a perfect fit and 0 means no better than predicting the mean.
Characterizing Nonlinear Dynamics via Smooth Prototype Equivalences warps experimental data onto hand-chosen analytic prototypes. It reports that a mismatched prototype produces high-variance fits. This constrains us: freezing $K$ is only safe when the simulator's topology is right, so our protocol must detect that case rather than assume it.
The evaluation methodology comes from two further papers.
- On the Equivalency, Substitutability, and Flexibility of Synthetic Data defines the equivalency ratio (about 30 synthetic frames per real frame for tracking) and substitutability (synthetic pretraining plus 20% real data nearly matches 100% real on MOT17). It measured these only for black-box trackers and only as whole-model numbers. We measure them per component.
- Is this Model for Real? Simulating Data to Reveal the Proximity of a Model to Reality tests whether fitted parameters are real or artifacts: it simulates from the fitted model, refits, and compares. It showed that parameter values once dismissed as simulation artifacts occur in real fits. We apply this test to $\omega_j, \rho_j$ to decide whether an explanation is trustworthy.
The concept-bottleneck line, Concept Bottleneck Models and This looks like that, frames interpretability as a lossy bottleneck. Our setting differs: the warp is lossless, and the interpretable object is the latent operator, not a compressed concept vector. We borrow only the idea of intervention. Freezing $K$ at the simulator's values is a test-time intervention on the interpretable layer.
Research Program
The work has three stages.
Controlled testbed. We generate a "source" simulator and several "target realities" whose gap has a known location. Geometry-only gaps change the observation map: a monotone nonlinear rating curve, sensor saturation, and coloured or heavy-tailed noise, in the spirit of the non-Gaussian failures noted in One or two things we know about concept drift. Dynamics-only gaps shift a frequency or decay rate. Topology gaps add a second attractor, which no fixed $K$ can represent. For each target we run all three adaptation regimes and sweep $\lambda$ and the fraction of real data from 5% to 100%. Because the ground truth is known, we can check directly whether the gap attribution is correct.
Real systems. We apply the protocol to two real datasets with simulated counterparts.
- Muddy Creek streamflow, pretrained on LDDMD's synthetic streamflow generator.
- LASA handwriting motions, pretrained on synthetic stable ODE trajectories of the kind used by Zhi et al.
On each we report the attribution, the substitutability curve, and the parameter-authenticity test. Muddy Creek is the headline case because the known 0.46-versus-0.89 gap gives the question weight.
Noise. Since LDDMD's real-data failure is attributed to high-frequency noise, we add one component inside the geometry side: an explicit observation-noise term with a learned spectrum, adapted along with $\phi$. This lets the warp-only regime model sensor noise instead of bending $\phi$ to fit it. It is ablated separately.
Baselines
The naive reference points are LDDMD trained from scratch on each real-data fraction and an LSTM trained from scratch on the same fractions. These show what synthetic data buys at all. On the full Muddy Creek record, the reported numbers are NSE 0.46 for LDDMD and NSE 0.89 for the tuned LSTM (LDDMD). We will first reproduce both numbers and only then report our own.
The strongest competitor on accuracy is an LSTM pretrained on the same synthetic trajectories and fine-tuned on real data. This is the black-box recipe whose substitutability Chang et al. documented for tracking. No reported result was found on Lacuna for synthetic-pretrained LSTMs on Muddy Creek, so we establish that number ourselves. Every model gets the same synthetic corpus, the same real splits (including a held-out extrapolation window beyond the training horizon), the same number of tuning trials per model, and matched wall-clock budgets.
The second strong competitor is full fine-tuning of the pretrained LDDMD, which corresponds to $\lambda = 0$ with both components free. This baseline isolates the key ingredient. If split transfer does not beat it, the component separation adds nothing.
Two further ablations complete the comparison. Adapting $K$ alone checks the attribution from the opposite direction. The split model without the explicit noise term isolates the noise component.
Split transfer counts as beating the baselines under three conditions:
- On geometry-gap targets and on real data where it attributes the gap to geometry, it reaches the full fine-tuned LDDMD's NSE with at most half the real data.
- It keeps the frequencies within the synthetic recovery tolerance.
- On the extrapolation window, it matches or exceeds the synthetic-pretrained LSTM.
We do not expect it to match the LSTM's 0.89 in-horizon, and we will report that honestly as a trade-off.
Why not just…?
Why not just pretrain an LSTM on simulations and fine-tune it? For in-horizon accuracy this is a genuine threat, and it may win. LDDMD's own results show a tuned LSTM leading in-horizon on real streamflow but diverging when extrapolating, where the structured model holds trends. The fine-tuned LSTM also cannot say whether the simulator's physics or its sensor model was wrong. The extrapolation window and the attribution experiments settle this. If the LSTM also extrapolates well after synthetic pretraining, the accuracy argument for our approach weakens, and only the diagnostic value remains.
Why not just fine-tune the whole interpretable model? Full fine-tuning lets the warp and the operator trade off against each other. A shifted frequency can be partly faked by a distorted warp, so the adapted $\omega_j$ may look plausible while being an optimization artifact, exactly the failure the authenticity test in Is this Model for Real? is built to catch. The controlled testbed shows this directly: on geometry-only gaps, full fine-tuning should drift $\omega_j$ away from the true unchanged value while warp-only fine-tuning keeps it. If full fine-tuning does not drift, our authenticity argument fails.
Why not just add a black-box residual network to the interpretable model? A residual model will likely recover much of the real-data accuracy. The cost is an explanation that covers only part of the signal. We will run it as an extra comparison and report the fraction of real-data variance carried by the residual. If that fraction is small and accuracy is high, the residual hybrid is the simpler answer, and we will say so.
Why not just use DMD or SINDy on the real data? Smooth Prototype Equivalences reports SINDy degrading in low-sample, high-noise regimes. LDDMD reports that plain DMD cannot model memory. Low data and heavy noise are exactly our regime, so both methods serve only as reference rows.
Decisive Checks
Three experiments decide the proposal.
Attribution. On the controlled testbed, the adaptation regime that gains most must match the true location of the gap in at least most of the targets. For topology gaps, where neither frozen component can fit and the prototype-mismatch warning from Smooth Prototype Equivalences applies, all regimes should fail visibly. Silent success on a topology gap would falsify the diagnostic.
Substitutability. On Muddy Creek and LASA, we plot NSE (or trajectory MSE for LASA, where no reported accuracy result was found on Lacuna) against the fraction of real data for every regime. We then read off the real-data fraction at which each regime reaches the full-data fine-tuned score, which is the component-level version of Chang et al.'s substitutability. The headline claim holds if warp-only adaptation reaches that score with less real data and keeps authentic $\omega_j$.
Authenticity. We simulate from each adapted model, refit on the simulations, and check whether the refit $\omega_j, \rho_j$ fall inside the spread of fits to real data. The proposal fails if warp-only models pass the accuracy check but fail this test, because that would mean freezing the physics merely hid the mismatch inside the warp.
References
- Learning Efficient and Robust Ordinary Differential Equations via Invertible Neural Networks
- Latent Diffeomorphic Dynamic Mode Decomposition
- Learning Dynamics Models with Stable Invariant Sets
- Characterizing Nonlinear Dynamics via Smooth Prototype Equivalences
- On the Equivalency, Substitutability, and Flexibility of Synthetic Data
- Is this Model for Real? Simulating Data to Reveal the Proximity of a Model to Reality
- Concept Bottleneck Models
- This looks like that: deep learning for interpretable image recognition
- One or two things we know about concept drift - a survey on monitoring in evolving environments. Part A: detecting concept drift