Protein behavior rarely decomposes into independent pairs. When residue A hydrogen-bonds to residue B, B's electronic structure changes, which changes how B interacts with residue C — a genuinely three-body effect. The best current models acknowledge this: H3-DDG builds explicit three- and four-body interaction layers to predict how mutations change binding free energy, and beats pairwise models by large margins when several mutations interact. But these models face a hard combinatorial wall. Enumerating all $k$-residue tuples in a protein of $N$ residues scales as $O(N^k)$, so every existing architecture strangles the higher-order layers with fixed spatial cutoffs — hyperedges only within 5 Å in TSMGen, four-body attention only within an 8 Å sphere in H3-DDG. These static balls are exactly where non-local cooperative effects, the allosteric chains that biology actually uses, get cut off.
Meanwhile, an unrelated literature has solved a structurally identical problem. Dynamic Sparse Training (DST) trains neural networks that never hold more than a small fraction of their possible weights at once: every few hundred iterations, the least useful connections are pruned and the freed budget is spent on new connections chosen by gradient magnitude. Liu et al. showed why this works — the model doesn't need all parameters simultaneously, it needs to visit enough of them over time (in-time over-parameterization), and sparse ResNets trained this way match dense baselines at extreme sparsity. Nobody has applied this temporal-exploration mechanism to the interaction structure of a biomolecular model rather than to its weight matrices. This proposal does exactly that: we replace the static spatial balls that define which residue groups a many-body model can see with a dynamically rewired budget of hyperedges, so the model learns where the non-local cooperativity is instead of being told it cannot exist beyond 8 Å.
Core Claim
Fix a per-residue budget $B$ of active hyperedges, where a hyperedge is a small group of jointly-attending residues (a 3- or 4-tuple). At any training step $t$ only these hyperedges are instantiated, so compute and memory stay flat in $N$; but the set membership evolves during training under a prune-and-regrow cycle, so the cumulative set of explored tuples,
$$R_s = \frac{|\bigcup_t E_{\text{active}}(t)|}{|E_{\text{full}}|},$$
approaches 1 even though the instantaneous hyperedge count never exceeds $B$. The claim is that gradient-guided hyperedge rewiring will (a) recover the accuracy of static-cutoff models at equal or lower cost, and (b) discover non-local cooperative tuples — pairs of residues tens of Ångströms apart jointly participating in one hyperedge — that fixed 8 Å balls are provably blind to, improving prediction on multi-point mutations where epistasis between distant sites dominates. This is the mechanism of In-Time Over-Parameterization, which showed sparse ResNet-50s matching dense accuracy at 90% sparsity purely by temporal exploration, transplanted from weight masks to the higher-order interaction hypergraph of a structural predictor.
The distinction from the nearest prior work matters and is sharp. H3-DDG (Accurately Predicting Protein Mutational Effects via a Hierarchical Many-Body Attention Network) instantiates three- and four-body layers inside fixed metric balls chosen a priori; its own authors flag the $O(N^4)$ trap as the reason. Our hyperedges have no radius constraint and move during training. Exploring the impact of adaptive rewiring in Graph Neural Networks rewires during training, but only the weight matrices of ordinary pairwise message passing — its data graphs and its interactions stay strictly two-body, and it reports that weight sparsity above roughly 30% degrades GNN accuracy. We rewire the interaction structure itself and keep weights dense. Energy Loss Functions for Physical Systems / Kaba et al. sparsify molecular interaction graphs to $O(N)$ via rigidity theory, but their graph is fixed a priori from geometry, whereas ours is learned. And Hypergraph Dynamic System makes hypergraph propagation stable with ODE formulations, but its hyperedges are static and its setting is citation networks, not 3D macromolecules.
Research Program
We build DHT (Dynamic Hyperedge Training) on a H3-DDG-like backbone, because that architecture is the only one in the file with a clean separation between pairwise residue graphs and explicit $k \ge 3$ layers, and it comes with reported numbers we must beat. The pairwise layer stays dense and standard. All novelty sits in the higher-order layer.
The mechanism has four parts. First, hyperedge scoring and pruning: each active hyperedge carries an importance score accumulated over an update interval $\Delta T$ from its contribution to the loss (measured by the magnitude of gradients flowing through its aggregation gate, in the spirit of RigL's gradient-magnitude regrowth). Every $\Delta T$ steps, the lowest-scoring fraction is dropped. Nowak et al. and Liu et al. both showed connections need this settling period before their value can be judged, so $\Delta T$ is a first-class object of study, not a nuisance hyperparameter. Second, candidate-pool regrowth: naïvely regrowing over all $O(N^k)$ tuples is intractable, so each residue keeps a precomputed candidate pool of potential hyperedge partners assembled from cheap signals — residues within a generous shell (say 20 Å, far beyond any existing cutoff), strongly co-mutating sequence positions, and members of its secondary-structure and contact-map neighborhood. Regrowth allocates the freed budget to candidates with the largest estimated gradient magnitude, evaluated in a single extra forward pass per cycle rather than per-step. This makes the exploration non-local in content while staying local in cost: the pools are cheap to build, and only $B$ hyperedges are ever instantiated. Third, sparsity-aware initialization: Evci et al. proved that standard initializations fail under sparsity because heterogeneous fan-in distorts signal variance, and this has never been analyzed for hypergraph message passing, where a residue's fan-in counts whole tuples. We derive and test the analogous fan-in-corrected initialization for hyperedge aggregation — this addresses a genuine open problem, since Hypergraph Dynamic System shows deep hypergraph networks already teeter on gradient collapse even when dense. Fourth, rewire-annealing: following the adaptive-rate and early-stopping findings of Cambier van Nooten et al., the rewiring fraction decays and eventually freezes, locking in a final learned hyperedge topology that becomes part of the deployed model, so inference cost equals a static-sparse model.
The scientific payoff beyond accuracy is interpretation: the final hyperedge set is a predicted map of higher-order cooperativity. We will test whether the learned non-local hyperedges (partners farther than 10 Å apart) concentrate on residues that mutation databases flag as epistatic, and on documented allosteric pathways — the regime where Lu et al. and Bryant et al. showed assembly prediction fails precisely because distal, induced cooperativity cannot be modeled locally. Najmanovich et al. proved that pairwise contact potentials cannot uniquely identify native states against decoys; a learned sparse hyperedge structure is a concrete, testable hypothesis about the minimal set of many-body terms needed to fix that.
Baselines
The strongest existing competitor is H3-DDG (Accurately Predicting Protein Mutational Effects via a Hierarchical Many-Body Attention Network), which reports a Pearson correlation of 0.75 on SKEMPI v2, beating the previous best (BA-DDG) by over 10% on multi-point mutations, and outperforming Prompt-DDG by 62.61% and BA-DDG by 34.26% on BindingGYM. It is the strongest because it is the current best on exactly our task with exactly our architecture class; we reimplement its backbone ourselves and train both H3-DDG (static 8 Å balls) and DHT on identical data, identical 3-fold structure-split cross-validation on SKEMPI v2, identical pre-training, and an identical FLOP-and-wall-clock budget, so the only difference is how the hyperedge set is chosen. Beating it means a higher Pearson correlation on SKEMPI v2 than 0.75 and a higher Spearman on BindingGYM's multi-point and Inter-Assay splits, at equal or lower training cost per epoch. The supervised BA-DDG line (Boltzmann-Aligned Inverse Folding Model as a Predictor of Mutational Effects on Protein-Protein Interactions, Spearman 0.5134 on SKEMPI v2) and its unsupervised BA-Cycle variant (Spearman 0.3201) serve as secondary reference points for the thermodynamic-cycle framing, though they are weaker on this task. The naive reference point is the pairwise-only ablation of the same backbone: H3-DDG's own ablations show that removing the explicit 3- and 4-body layers costs 12–34% on multi-point mutations, so a pairwise-only model is the floor that establishes the many-body layers matter at all. The key ablation isolating our ingredient is a static large-radius variant: identical backbone, identical average hyperedge count, but hyperedges drawn once from the same 20 Å candidate pools and never rewired. If dynamic rewiring beats this, the win comes from temporal exploration, not merely from a bigger radius; if it doesn't, the proposal's mechanism is not the source of any gain and we will say so. We also run random-regrowth (SET-style) and fixed-rate rewiring as ablations of the gradient-guided and adaptive components respectively, since Cambier van Nooten et al. report fixed-rate rewiring underperforming adaptive rewiring on MUTAG (0.90 vs. lower for fixed-rate, with GINE) and we expect the same ordering here.
Why not just…?
Why not just use static spatial cutoffs (k-nearest-neighbor or radius graphs)? This is the default in the field — 5 Å in TSMGen, 8 Å in H3-DDG — and it is cheap and adequate for local chemistry. But it is blind by construction to exactly the effects that motivate higher-order modeling: Lu et al. and Bryant et al. document that assembly binding depends on distal conformational changes, and Najmanovich et al. prove pairwise locality cannot stabilize native states against decoys. Our decisive experiment is the static-20 Å ablation above: if a bigger static ball fully closes the gap to DHT, then radius was the answer and rewiring is unnecessary, and the paper's conclusion becomes "the cutoff was too small," which we would report honestly. The additional evidence that static sparsification underperforms dynamic is that static sparse networks get trapped in poor basins while rewiring boosts early gradient norms (How does topology influence gradient flow in sparse networks?) — though we note that evidence comes from weight masks, not hyperedges, which is precisely why the question needs the experiment.
Why not just train dense and prune after? Post-hoc pruning and Lottery-Ticket methods presuppose a converged dense model. For the weight matrices of our backbone that is feasible. But for the higher-order interaction structure, the dense object being pruned is an $O(N^k)$ hypergraph that cannot be instantiated at all — HierAFold (Efficient Prediction of Large Protein Complexes via Subunit-Guided Hierarchical Refinement) exists precisely because dense token interaction crashes GPUs beyond 4,000–5,000 tokens (AlphaFold3 goes out-of-memory above 5,000), and H3-DDG confines its 4-body attention to 8 Å balls explicitly to dodge the $O(N^4)$ trap. You cannot prune a tensor that never fits in memory. DST's whole premise is never instantiating the dense model, which is why the transplant is necessary rather than decorative.
Why not just use deeper pairwise models? Stacking more two-body message-passing layers is the tractable textbook answer, and some higher-order effects do leak through depth. Three results in the file say it is not enough: Najmanovich et al.'s impossibility result for pairwise contact potentials; H3-DDG's ablations showing explicit 3- and 4-body layers contribute a 12–34% boost on multi-point mutations where pairwise layers stall; and the over-smoothing wall — standard hypergraph networks collapse from roughly 70% to 35% accuracy as depth grows to 64 layers, and only ODE-stabilized variants survive (Hypergraph Dynamic System). Depth is a poor substitute for explicit tuples when the tuples are the physics. Our pairwise-vs-many-body ablation re-confirms this on our own backbone before any rewiring claim is made.
Why not just do random rewiring on a fixed schedule (standard SET)? This is the cheapest version of our own idea, so it is a real threat rather than a straw man. The evidence against it is comparative: Cambier van Nooten et al. found fixed-rate rewiring consistently below adaptive rewiring across MUTAG, PROTEINS, and the N-1 grid task, with random exploration destabilizing late training, matching the settling-period analyses of Nowak et al. and Liu et al. Gradient-guided regrowth concentrates the budget on tuples that are actually moving the loss. We settle it directly: random-regrowth DHT at matched budget is one of our ablations, and if the gap to gradient-guided regrowth is within noise, we will conclude the exploration schedule doesn't matter and report that the non-local candidate pools were the operative ingredient.
Proposed Experiments and Decisive Checks
The primary benchmark is SKEMPI v2 (over 7,000 mutations across 348 complexes, 3-fold cross-validation split by structure, following both H3-DDG and BA-DDG protocols), reporting Pearson correlation, with the number to beat being H3-DDG's 0.75; we also evaluate robustness using AlphaFold3-predicted rather than crystal structures, where H3-DDG reports only a 5.4% performance drop, a bar for deployment realism. The second benchmark is BindingGYM (roughly 500,000 curated DMS points from 25 assays, with Central-vs-Extremes and Inter-Assay splits), reporting Spearman on multi-point and inter-assay settings, where H3-DDG's relative margins over Prompt-DDG (62.61%) and BA-DDG (34.26%) define the comparison set; the file gives no absolute BindingGYM correlation for any method, so we will treat our own retrained H3-DDG as the absolute reference on that benchmark. Third, a scaling probe on the large-assembly setting of HierAFold (complexes of 5,000–10,000 tokens, 664 interfaces), where we test whether the flat hyperedge budget lets many-body interaction modeling operate at token counts that dense higher-order attention cannot touch — HierAFold reports 73.1% success above 5,000 tokens and about 25 GB (~40%) peak-memory savings, and AlphaFold3 crashes outright, so success there means many-body terms surviving into a regime currently restricted to pairwise heuristics.
Three decisive checks go beyond headline metrics. The exploration check plots cumulative coverage $R_s$ against final accuracy across rewiring fractions and $\Delta T$ values; the ITOP prediction is a rising curve that saturates, and if accuracy is flat in $R_s$, temporal exploration is doing nothing and the mechanism is falsified. The locality check measures what fraction of final frozen hyperedges connect residues more than 10 Å apart, and whether ablating them at test time hurts more than ablating matched local ones; the allosteric prediction is that non-local hyperedges are disproportionately load-bearing on multi-point mutations specifically. The stability check trains deep variants (16–64 hypergraph layers) with and without our fan-in-corrected initialization, against the dense-collapse baseline documented in Hypergraph Dynamic System — establishing whether sparsity-aware initialization generalizes from weight masks to hyperedge aggregation, a question no paper in the file has touched. One honest limitation we flag up front: hyperedge-level gradient estimation during regrowth is approximate (one extra forward pass per cycle), and if that approximation proves too noisy on 348-complex SKEMPI, we will fall back to candidate scoring from activation statistics and say which variant produced each number. Every comparison is run at matched training FLOPs and wall-clock, because a method that wins by spending more compute has won nothing.
References
- Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training
- Accurately Predicting Protein Mutational Effects via a Hierarchical Many-Body Attention Network
- Exploring the impact of adaptive rewiring in Graph Neural Networks
- Energy Loss Functions for Physical Systems
- Hypergraph Dynamic System
- Boltzmann-Aligned Inverse Folding Model as a Predictor of Mutational Effects on Protein-Protein Interactions
- Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win
- Efficient Prediction of Large Protein Complexes via Subunit-Guided Hierarchical Refinement