Which Sensors Should the Correction Network See? Federated Feature Search for Physics-Plus-Residual Models

Many working prediction systems in engineering and medicine are hybrids. A physics or trend model gives the backbone of the prediction, and a small learned model corrects what the physics misses: a residual network inside a vehicle simulator, a correction module on a blood-flow simulator, a neural net on top of a Grey trend model. The correction model is usually given every available sensor channel, and in deployment those channels are spread across sites that cannot pool their data. We propose a way for sites to jointly decide which input features the correction model should use, by exchanging only one score per candidate. The search judges each feature by whether it explains what the physics model gets wrong, not by whether it predicts the target, so channels that only repeat information the physics already uses are dropped.

The second idea is a new way to steer the search. A search over feature subsets has to trade off two things: trying distant, untested subsets (exploration) and refining the subsets that already score well (exploitation). Most methods control this trade-off with a fixed schedule over time. In federation there is a better control signal: whether the clients agree on which candidate subsets are good. When clients rank the candidates the same way, the search should narrow and refine. When their rankings conflict, the search should widen. If the conflict persists, the clients are likely asking different questions, and the population should split into client groups with their own subsets. In two sentences: sites rank candidate feature masks for the hybrid model's correction term using a bounded "skill over physics" score, and the server evolves the masks with a mutation rate set by how much the sites' rankings disagree. Lasting disagreement triggers per-group masks instead of one forced consensus.

Core Claim

In federated hybrid modeling, three choices should beat both fixed-schedule federated search and gradient-based sparsity at equal communication. The first is to define redundancy relative to the physics prediction. The second is to evaluate masks with a bounded score that clients return as a single number. The third is to adapt exploration from cross-client rank agreement. The advantage should be largest when the data differ across clients and when the correction learner cannot be differentiated, for example a random forest. We expect smaller and more stable masks, lower error than training the correction model on all features, and less negative transfer than a single global mask.

Relation to Prior Work

Several papers motivate the gap. Federated Residual Learning splits a prediction into a shared global model plus a per-client residual model and removes negative transfer on letter and pendigits. However, the system designer splits the features into global and local by hand. Our method learns that split, and its baseline term is mechanistic rather than a statistical global model. FedGA-Tree supplies the protocol we reuse: the server keeps a population of structures, and clients fit them locally and return only scalar fitness. It uses fixed crossover and mutation rates. Its authors also report that unbounded mean squared error lets a single outlier client distort the population. Our bounded skill score and our agreement-driven rates address both problems directly. Distributed Genetic Algorithm for Feature Selection shows that genetic search over binary masks removes 50% distractor features on the Agnostic datasets. It runs on a shared cluster, not under privacy constraints.

Other papers constrain the design. Secure and Lightweight Feature Selection for Horizontal Federated Learning makes federated filter ranking fast and secure, but it scores features one at a time. That is exactly the case that fails when a feature matters only in combination with others, or only conditional on the physics output. Federated Hybrid Model Pruning through Loss Landscape Exploration prunes weights inside networks to reduce variance across non-IID clients. It does not address input channels. Hybrid Federated Learning for Feature & Sample Heterogeneity aligns clients that hold different feature sets, but it keeps every feature and uses hand-tuned regularizers.

On exploration control, the metaheuristic literature adapts exploration in three ways. The first is time schedules: linear or nonlinear inertia decay (Zhang et al.) and cosine factors as in Improved Barnacle Mating Optimizer. The second is population-diversity feedback (Applying Fusion of Pso-Abc Algorithm on the Minimax Location Problem). The third is fitness-scaled weights (Improved Grey Wolf Optimization Algorithm and Application). All three treat the objective as a single function. None can tell "the landscape is rugged" apart from "the clients disagree about the landscape," and that distinction is the one federation creates. We reuse the cosine and diversity controllers as ablations. Finally, the hybrid papers show why the residual target matters. Inferring Cardiovascular Biomarkers with Hybrid Model Learning and NeuralFMU both get large gains from a learned correction term. The cardiovascular work also reports that inference becomes unstable when correlated quantities interact, which suggests the correction model can absorb information it should not use.

Research Program

Each client $i$ holds a fixed physics or trend model $f_{\text{phys}}$ and trains a small correction learner $g$ on a feature mask $m \in \{0,1\}^D$, where $D$ is the number of candidate input features. The client scores mask $m$ on its own held-out data by the bounded skill over physics:

$$s_i(m) = \max\!\left(-1,\; 1 - \frac{\mathrm{err}_i(f_{\text{phys}} + g_m) + \lambda \|m\|_0 / D}{\mathrm{err}_i(f_{\text{phys}})}\right),$$

Here $\mathrm{err}_i$ is the client's validation error and $\lambda$ is a sparsity price. The score is at most 1, and it is clipped at $-1$ so that one badly fitting client cannot dominate the population. Because the correction model is trained on the physics residual, a feature whose information $f_{\text{phys}}$ already uses brings no gain and pays the sparsity price. This is the operational meaning of redundancy relative to physics.

Each round, the server sends $K$ masks, which costs $K \cdot D$ bits. Each client returns $K$ scalars. The server measures agreement among the clients' rankings of the $K$ masks using Kendall's coefficient of concordance $W_t \in [0,1]$ and sets the mutation rate as

$$p_t = p_{\min} + (p_{\max} - p_{\min})(1 - W_t).$$

High agreement narrows the search, and conflict widens it. If $W_t$ stays below a threshold for $R$ rounds, the server clusters clients by their score vectors and evolves a separate mask population for each cluster. Selection uses the median score across clients so that outliers have limited influence.

We will build this protocol in three settings. The first uses correction learners that can be differentiated (small MLPs). The second uses learners that cannot (random forests, matching the RVoG-plus-forest pattern in A Method for Forest Canopy Height Inversion). The third uses distractor-augmented inputs where the truly redundant features are known. We will also characterize when splitting helps and when it fragments the data. Two costs are stated up front. Local compute grows with $K$ correction fits per round. Mask scores leak which features help each site; we treat that leak as a measured quantity, not a solved problem.

Baselines

The naive reference points are three. The first is the physics or trend model alone. The second is the hybrid with an all-feature correction model trained by FedAvg (federated averaging of client weights). The third is the hybrid trained purely locally at each client. All three use the same client partitions, the same correction architecture, and the same number of training rounds. Beating them means lower test error than the all-feature federated hybrid and no client worse than its local-only model.

The strongest gradient-based competitor is a federated group-lasso gate: a learnable sparse gate on each input channel of an MLP correction model, trained with FedAvg and with its sparsity strength tuned on the same validation budget we give $\lambda$. It is the strongest competitor because it directly attacks feature redundancy end to end. We will compare at equal uplink bytes and equal client compute. Beating it means lower error, or equal error with fewer bytes, on the differentiable setting. On the non-differentiable setting the gate does not apply, so there we compare against the importance-based rankings described next.

The strongest filter-style competitor is a plaintext version of SeiFS-style federated ranking from Secure and Lightweight Feature Selection for Horizontal Federated Learning, keeping the top-$k$ features. We run two variants. One ranks features by mutual information with the raw target, as the original method does. The other ranks by mutual information with the physics residual, which isolates the value of the residual criterion from the value of combinatorial search. Federated random-forest importance ranking serves as the analogous baseline for tree correction models.

The strongest federated-search competitor follows FedGA-Tree's operators, from FedGA-Tree: fixed mutation and crossover rates, elitism, and a raw mean squared error fitness. It uses the same $K$, the same rounds, and the same client participation rate. FedGA-Tree reports an average 5.58% F1 gain over local models on its 28-dataset tabular suite. Since that result is on decision-tree structures, not hybrid masks, we use it as a sanity target only on the tabular datasets we share with it, adult and creditcard.

We also run FedRes (Federated Residual Learning) with its hand-designed global/local split on the LIBSVM suite. No per-dataset numbers for it appear in the file, so we rerun it ourselves.

Three ablations isolate the key ingredient. The first replaces agreement-driven mutation with a cosine schedule, as in IBMO. The second replaces it with population-diversity feedback, as in PSO-ABC. The third removes cluster splitting. A centralized genetic search over pooled data, in the style of Distributed Genetic Algorithm for Feature Selection, is the upper reference. It reports MLP F1 rising from 0.875 with all features to 0.889 on Gina Agnostic.

Why not just…?

Why not just put a group-lasso gate on the inputs and run FedAvg? This is the real threat. For a differentiable correction model, a learned sparse gate may match combinatorial search at lower cost. The comparison at equal bytes on the differentiable setting will settle it. If the gate wins there, we will narrow the claim to non-differentiable correction learners and to the scalar-only communication regime. That regime matters in practice: the hybrids in Integrated Wavelet-Grey-Neural Network Model and the RVoG-plus-forest pipeline use components with no usable gradient, and returning $K$ numbers per round costs far less than returning model weights. The gate also yields one global sparsity pattern, so it cannot by itself discover that two groups of clients need different channels.

Why not just train the correction model on all features and rely on regularization? The feature-redundancy literature reports that extra dimensions actively hurt. Greedy eigencomponent selection shows accuracy rising, plateauing, and then falling as components are added on MVTec-AD (Gaussian Image Anomaly Detection with Greedy Eigencomponent Selection). EEG channel selection raises accuracy from 78.26% to 81.38% and lowers the spread across subjects (An EEG Channel Selection Framework for Driver Drowsiness Detection). Our all-feature federated hybrid baseline tests directly whether the same holds for the correction term.

Why not just rank features one at a time against the physics residual? It is cheap and may capture most of the gain. The residual-target filter baseline is included exactly to measure this. On the distractor datasets and the sensor-network lags, where features interact (neighbor-node readings are informative only jointly), we expect subset search to win. If it does not, the bounded residual criterion is the real contribution and the search machinery is unnecessary.

Why not just use an existing time-based schedule such as a cosine factor? Time schedules cannot distinguish a hard landscape from disagreeing clients. The schedule ablations measure whether the agreement signal adds anything beyond them, and the effect should grow with client heterogeneity.

Proposed Experiments

The first testbed is controlled. We use the Sylva, Gina, and Hiva Agnostic datasets, which have 50% known distractors. We split them across 10–50 clients under IID and Dirichlet-skewed partitions, and add a synthetic "physics" term that explains a known subset of the true features. We measure Jaccard overlap with the true non-redundant set, F1 against the centralized reference of 0.889 on Gina, and how often masks wrongly include features the physics term already explains.

The second testbed is a hybrid. We use PulseDB with subjects grouped into clients. The physics-only baseline gives a heart-rate relative mean absolute error of 10.88%, and the centralized hybrid gives 6.84% (Inferring Cardiovascular Biomarkers with Hybrid Model Learning). Candidate features are hand-derived waveform descriptors plus correction-module latents. Success means approaching 6.84% federatedly with a smaller input set. No federated result on PulseDB was found on Lacuna.

The third testbed treats each node as a client. If the 50-node environmental sensor-network data are obtainable, each node is a client, the baseline is a trend model, and the candidate features are lagged readings from neighboring nodes. The centralized CNN-BiLSTM reports 19.3%–32.7% error reduction over its baselines (Environmental Prediction Using a Spatiotemporal WSN). No federated result was found on Lacuna. We also run MDDD EEG with one subject per client as a heterogeneity stress test, where the centralized channel-selection method reports 81.38% (no federated result found on Lacuna), and the LIBSVM suite for the FedRes comparison.

Metrics are test error, error at the worst-off client relative to local-only training, mask size, mask stability across random seeds, uplink bytes, and client compute.

The decisive checks are three. The first is whether agreement-driven control beats both the cosine and diversity controllers by a margin that grows with Dirichlet skew. If the margin is flat, the federated signal adds nothing. The second is whether the residual-target criterion removes physics-explained features that raw-target ranking keeps. The third is whether the method beats the group-lasso gate at equal bytes in the non-differentiable setting while staying within noise of it in the differentiable setting. Losing badly to the gate in the differentiable setting narrows the claim as described above.