When engineers fit a physics model to data from one machine, and the model's equations are subtly wrong (say, a spring treated as linear when it stiffens at large deflections), the fitting procedure quietly bends the physical constants to hide the error. The constants come out biased, and the model breaks as soon as the machine enters a new operating regime. This proposal argues that a fleet of similar machines, trained together without pooling raw data, can see through this in a way no single machine can. Different machines run under different loads and speeds. A missing term in the equations shows up consistently across those regimes, while a wrong constant shows up differently. Federated learning, where many devices jointly train one model by sharing only model updates, can therefore do more than average away noise. If the shared model is built correctly, federation can supply the variety of operating conditions needed to tell "the equations are incomplete" apart from "this machine's constants differ."
Concretely, we propose a federated gray-box identification method. A gray-box model is a known physics equation plus a learned correction term. In our version, each client's dynamics are split into three parts: nominal physical parameters shared by the fleet, a missing-physics correction shared by the fleet, and a small client-specific correction. The key constraint is that both corrections must be orthogonal to the directions the nominal parameters can already explain. Those directions are estimated from the pooled operating conditions of all clients, and the pooling happens through summed sufficient statistics rather than raw trajectories. Our central, testable claim is that under this construction, bias in the physical parameters shrinks as the fleet's operating diversity grows. Plain federated least squares behaves differently: its bias from model-form error does not shrink as clients are added, no matter how many there are.
Where the gap is
Federated system identification is now well posed for correct model structures. FedSysID identifies linear time-invariant systems across clients whose system matrices lie within distance $\epsilon$ of a nominal model. It proves error of order $C_1/\sqrt{MN} + C_2\epsilon$, where $M$ is the number of clients and $N$ the number of rollouts per client, and reports roughly a 10x error reduction for 100 clients relative to one. Federated Nonlinear System Identification extends this to dynamics that are linear in a hand-chosen dictionary of nonlinear features $\phi(x,u)$, with $1/\sqrt{M}$ scaling on pendulum and quadrotor fleets. Both papers motivate this proposal and constrain it. They show that federation buys sample efficiency, but both assume the true equations lie inside the model class. When they do not, the $C_2$ floor mixes genuine client heterogeneity with systematic model-form bias, and nothing in either method separates the two.
The structural-uncertainty literature supplies the missing ingredient but works on a single machine. The physics-integrated hybrid framework treats model-form error as an unmeasured residual force. It estimates that force with dual Kalman-type filters and fits a Gaussian process (a flexible probabilistic regression model) from states to the force, extrapolating Duffing and Bouc-Wen responses 90 seconds ahead from 40 seconds of data. The discrepancy modeling framework distinguishes corrections to the rate equations from corrections to the outputs, and notes that separating discrepancy from noise becomes unstable at high noise. Transformed Gaussian processes choose additive or multiplicative discrepancy through a link function and point out that this choice requires domain heuristics. We reuse these methods for the correction term.
The most important constraint comes from the sloppy-systems study. There, designing experiments to make poorly determined parameters identifiable pushed an EGFR signaling model into regimes where its simplifications failed: best-fit error rose from about 7,000 to over 96,000, and a DNA-repair systematic-error indicator rose from 0.76 to 2.0. We read this as follows. Rich excitation reveals structural error, and it becomes harmful only when the model has nowhere to put that error except the parameters. A fleet provides rich excitation for free. Our method gives the revealed error somewhere to go.
On the federated side, BPFed splits neural network weights into shared early layers and personalized heads. It treats weights as Gaussian distributions and uses the previous round's posterior as the next round's prior, but chooses the split by network depth, which its authors acknowledge is heuristic. Our split is set by the physics: parameter-explainable directions, fleet-shared residual, and local residual. It is enforced by orthogonality rather than by architecture.
Core Claim
Write client $i$'s dynamics as
$$\dot x = f(x,u;\theta^* + \Delta\theta_i) + \delta_s(x,u) + \eta_i(x,u) + \text{noise},$$
where $f$ is the nominal physics, $\theta^*$ the shared physical parameters, $\Delta\theta_i$ a small client deviation, $\delta_s$ a missing-physics correction shared by the fleet, and $\eta_i$ a client-specific correction.
Without further restriction this decomposition is not identifiable. Any part of $\delta_s$ that lies in the span of the parameter sensitivities $J = \partial f/\partial\theta$ can be absorbed by shifting $\theta^*$. This is exactly the over-tuning failure. We therefore constrain $\delta_s$ and $\eta_i$ to the orthogonal complement of $\mathrm{span}(J)$, measured in the empirical inner product over the union of all clients' visited states.
The subspace where parameters and structural error cannot be told apart is the part of the missing term that looks like a parameter change on the data seen. That subspace shrinks as the pooled data covers more regimes. A single client with narrow excitation has a large confounded subspace. A diverse fleet has a small one. Our claim is therefore stated in terms of diversity, not only sample size: at fixed total data, parameter bias should fall as the spread of client operating regimes increases. We further predict that this diversity effect is absent for federated least squares without the correction terms.
Research Program
We build the corrections as Bayesian linear models over random Fourier features, a finite feature approximation to a Gaussian process. The whole model at each iteration is then linear in the unknowns: parameter increments, shared residual weights, and local residual weights, with a hierarchical Gaussian prior shrinking $\eta_i$ toward zero. For nonlinear nominal physics we linearize $f$ around the current parameter estimate and iterate, in Gauss–Newton fashion.
Each communication round, every client sends the server summed Gram matrices and cross-moments over its data. These are $J^\top J$, $J^\top\Phi$, $\Phi^\top\Phi$, and the corresponding products with the target, where $\Phi$ is the shared residual feature matrix. The server sums these statistics, computes the orthogonal projector for the pooled sensitivity span, and solves exactly for the posteriors of $\theta^*$ and $\delta_s$. Clients then fit $\eta_i$ locally against the projected residual. Because the statistics are additive, the federated solution for the shared part equals the centralized one, which gives us a clean oracle to compare against. Communication cost is quadratic in the number of features and independent of trajectory length.
Following the discrepancy modeling framework, we also offer a sparse-dictionary variant of $\delta_s$ so that recovered missing physics can be read symbolically. Uncertainty comes directly from the Gaussian posteriors. As a second, distribution-free layer, we calibrate multi-step rollout intervals with the partial-exchangeability, quantile-sketch approach of federated conformal prediction. That paper was evaluated only on image classification, so applying it here tests whether it transfers to trajectories. It is not a contribution of its own.
On the analysis side, we will extend the FedSysID-style error decomposition. The aim is a bound whose model-form term depends on the norm of the missing physics projected onto the pooled sensitivity span, a quantity that shrinks with fleet diversity. We expect this analysis to be tractable only for linear-in-parameters nominal models, and we will say so. Three limitations are known in advance. We assume states and derivative estimates are available up to noise, and partial observation is left to local filters as in the hybrid framework. Summed Gram matrices are not formally private. Chaotic systems will limit rollout horizons regardless of method, as both SVISE and the discrepancy framework observe.
Baselines
The naive reference is local isolated calibration. Each client fits the nominal model's parameters to its own data by least squares, with no correction term and no federation. It sets the floor for sample efficiency and parameter bias. It is run with the same nominal model and the same data per client. Beating it on parameter error at small $N$ is expected and not by itself interesting.
The strongest existing federated competitor is dictionary-based federated least squares as in Federated Nonlinear System Identification, which reduces to FedSysID on linear systems. It is the only method in the file that federates physical identification with guarantees, and it is what practitioners would deploy. We run it in two forms. The first uses the nominal dictionary only, which absorbs model-form error into the parameters. The second uses an expanded dictionary with the same feature budget as our shared residual. Both receive the same clients, rollouts and communication rounds, and regularization is tuned on the same held-out client trajectories. For reported numbers, FedSysID reports about a 10x reduction in parameter estimation error for 100 clients versus one on its synthetic LTI fleet. For the nonlinear paper there is no reported numerical parameter error on Lacuna, only confirmation of $1/\sqrt{M}$ scaling. We will therefore reproduce both on their own synthetic, pendulum and quadrotor settings before adding structural error. Beating them means lower parameter bias that keeps falling with fleet diversity where theirs plateaus, together with better rollout error on held-out operating regimes, at equal communication.
The personalized-learning competitor is a BPFed-style model (BPFed). It keeps the nominal physics and adds a Bayesian neural residual whose early layers are federated and whose head is local. Its hyperparameter budget matches ours. It tests whether the split by network depth already captures the shared-versus-local structure. Beating it means comparable or better forecasts together with markedly less bias in $\theta^*$ and correct attribution of the shared missing term.
The centralized oracle pools all raw data and fits our model directly. It is included to confirm that federation loses nothing on the shared part, as the additive statistics guarantee.
Three ablations isolate the key ingredient. The first removes the orthogonality projection, so the corrections can trade off against the parameters. The second removes $\delta_s$, leaving only local corrections. The third merges $\delta_s$ and $\eta_i$ into one per-client residual. The first ablation is decisive. If it matches the full method on parameter bias, the proposal's main mechanism is not doing the work.
Why not just…?
Why not just expand the feature dictionary in federated nonlinear identification? This is the most serious threat. If the missing term happens to lie in a sufficiently rich dictionary, federated sparse regression will find it, and our method adds little beyond uncertainty estimates. Two issues remain even then. A rich dictionary contains many terms that are nearly collinear with the nominal parameter sensitivities, so the regression can still assign the missing physics to shifted constants. And that regression cannot separate fleet-wide missing physics from one client's local defect. The expanded-dictionary baseline with a matched feature budget settles this directly. If it matches us on parameter bias across diversity levels, we will report that the orthogonality projection, not the dictionary, is the only possible contribution, and test it alone.
Why not just learn a personalized neural residual on top of the physics? Such a residual will usually forecast well in the regimes it was trained on. Nothing stops it from absorbing a fraction of the parameter signal, however, leaving $\theta^*$ biased in the same way over-tuning does. The sloppy-systems results (sloppy systems) and the Bayesian optimization finding that a wrong model class converges to poor solutions regardless of data volume (model uncertainty in BO) suggest that this bias matters once the system is pushed off-distribution. Our held-out-regime rollouts and the parameter-bias metric expose this. If the neural residual extrapolates as well as we do, we concede the forecasting claim and retain only the parameter-attribution claim.
Why not just calibrate locally, since each machine is different anyway? Local fitting has no access to other clients' operating regimes. Its confounded subspace is therefore large, which is exactly the case where missing physics leaks into the constants. The diversity sweep below measures this directly, in addition to the known $1/\sqrt{MN}$ variance advantage of federation.
Why not just absorb the error into the existing parameters, as engineers usually do? The sloppy-systems and clutch-calibration studies (wet friction clutch) already document that this corrupts parameters and predictions. In our experiments this approach is the nominal-dictionary federated baseline, and we expect its bias not to shrink with $M$.
Proposed Experiments
We construct fleets of 10 to 100 clients on five systems. The first three are the 3-state linear system from FedSysID, and the pendulum and 13-state quadrotor from the federated nonlinear paper. The last two are the 2-DOF Duffing and 5-DOF Bouc-Wen oscillators from the hybrid framework, driven by varied random excitation and tested on El-Centro-style ground motion.
In every fleet, the simulated truth contains two kinds of term that the nominal model lacks. One is a fleet-shared omission, such as cubic stiffness, quadratic aerodynamic drag, or hysteresis treated as linear damping. The other is client-specific, such as a per-unit friction term, on top of bounded parameter deviations. We report four metrics: parameter bias $\|\hat\theta^* - \theta^*\|$, recovery error of $\delta_s$ on held-out states, multi-step rollout error in operating regimes no client trained on, and empirical coverage of rollout intervals from the posterior and from the federated conformal layer.
The decisive check is a diversity sweep at fixed total data. We hold $M \cdot N$ constant and vary how widely client excitation amplitudes and frequencies spread. Our claim predicts that parameter bias falls with diversity for the full method, stays flat for nominal-dictionary federated least squares, and stays flat for the ablation without the projection. A second check sweeps $M$ at fixed diversity, to confirm that the model-form term of the error floor does not decay with client count for baselines. A third check raises observation noise until the separation fails, locating the instability the discrepancy framework warns about.
The claim fails if any of these holds: the no-projection ablation matches the full method on bias, the expanded dictionary matches it across diversity levels, or bias does not respond to diversity at all.
References
- FedSysID: A Federated Approach to Sample-Efficient System Identification
- Federated Nonlinear System Identification
- Physics-integrated hybrid framework for model form error identification in nonlinear dynamical systems
- Discrepancy Modeling Framework: Learning Missing Physics, Modeling Systematic Residuals, and Disambiguating between Deterministic and Random Effects
- Transformed Gaussian Processes for Characterizing a Model's Discrepancy
- The Limitations of Model-Based Experimental Design and Parameter Estimation in Sloppy Systems
- Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representations
- Federated Conformal Predictors for Distributed Uncertainty Quantification
- State estimation of a physical system with unknown governing equations
- Exploring and Exploiting Model Uncertainty in Bayesian Optimization
- Inverse parametric uncertainty identification using polynomial chaos and high-order moment matching benchmarked on a wet friction clutch