Neural network force fields predict the energy of a group of atoms, and from that energy the force on each atom. People want to know how many atoms "interact at once" inside these models: whether the predicted energy behaves like a sum over pairs, over triplets, or over larger groups. The question matters because the answer describes the physics the model has learned, and because spurious large-group interactions are a plausible cause of simulations that blow up. Today the standard way to read off this "body order" is to delete atoms and re-evaluate the model on the smaller clusters that remain. This produced a striking result: trained models appear to "hallucinate" physically wrong decompositions, and forcing them to match the true decomposition hurts accuracy badly. We argue that atom deletion is the wrong experiment. It asks the model about isolated fragments it has never seen, which is exactly the off-distribution intervention that interpretability research warns can produce artifacts. We propose a different probe. Nudge a chosen set of atoms by tiny, thermally realistic amounts, separately and together, and measure the non-additive part of the energy change. This stays on the data manifold and has an exact physical meaning: the effect is identically zero for any set larger than the true body order.
The core claim is that body order should be defined and measured as an interventional quantity. It is a property of how the model's output responds to joint manipulation of atoms, not of how the model extrapolates to fragments. The proposal tests two things. First, whether the "body-order paradox" survives this on-manifold measurement. Second, whether spurious high-order response, measured this way, predicts which force fields fail in molecular dynamics better than static error does.
Core Claim
Let $E(\mathbf{x})$ be the model's predicted energy at configuration $\mathbf{x}$. Let $S$ be a set of distinct atoms, each assigned a displacement direction $\mathbf{u}_i$ with amplitude $\delta$. We define the interaction contrast as the inclusion–exclusion sum over all subsets $T \subseteq S$ of displaced atoms:
$$I_S(\mathbf{x};\delta) = \sum_{T \subseteq S} (-1)^{|S|-|T|}\, E\big(\mathbf{x} + \delta \textstyle\sum_{i \in T} \mathbf{u}_i\big).$$
Each term is a do-intervention that moves only the atoms in $T$. As $\delta \to 0$, $I_S/\delta^{|S|}$ becomes the mixed derivative of $E$ along the chosen directions, one derivative per atom in $S$. The key fact is elementary but exact. If the energy is a sum of terms each involving at most $n$ atoms, every term misses at least one atom of any set with $|S| = n+1$. Its mixed derivative over that set therefore vanishes. So $I_S \equiv 0$ for all sets larger than the true body order. A pair potential has zero three-atom contrast everywhere. A potential with angular three-body terms has zero four-atom contrast everywhere. Unlike sub-cluster energies, this test never moves the model off physically plausible geometries, provided $\delta$ is matched to thermal fluctuation amplitudes.
The novelty is not the derivative itself. It is the claim, which we will test, that this on-manifold interventional measure and the atom-deletion measure give different answers on trained models, and that the on-manifold measure is the faithful one. That reframes Resolving the body-order paradox of machine learning interatomic potentials. The paper trains SOAP-BPNN, MACE and PET on hydrogen octamers and finds they reach under 0.025 eV/atom RMSE while their sub-cluster energies disagree with the quantum many-body expansion. Adding sub-clusters to training to force agreement raised MACE's full-cluster error 3.4-fold. Two readings are possible. The models may genuinely compute energy through a non-physical decomposition. Alternatively, the deletion probe may measure extrapolation to unseen fragments rather than the model's mechanism on real configurations. If the second reading holds, the apparent conflict with the "appropriate decomposition" requirement for faithful explanations in Three Desiderata for Faithfulness in Machine Learning Explanations dissolves. A model could be physically decomposed on its data manifold and arbitrary off it, and training on fragments would hurt because it adds an extra off-manifold constraint.
Relation to Prior Work
The interventionist view of explanation supplies the method. The explanation game and Conceptual challenges for interpretable machine learning argue that explanations should answer "what happens to the output if we manipulate this variable." Andersen et al. add that interventions must be faithful to the mechanism. They also acknowledge that interventionist tools can certify several conflicting explanations at once. Our contribution is to choose an intervention whose meaning is fixed by physics, so the permissiveness problem is replaced by a checkable zero condition. We also adopt their concern about interventions that push internal states off-distribution, and turn it on the deletion probe.
Causal graph explainers constrain what we can borrow. Graph Neural Network Causal Explanation via Neural Causal Models (CXGNN) trains a neural causal model per candidate neighborhood and reports 100% motif match on BA+House and 66.67% on Benzene, where associational explainers scored at most 5% and 0%. OrphicX learns separate causal and background latent factors with backdoor adjustment, reaching 77.2% explanation accuracy on MUTAG versus Gem's 71.4%. Both explain discrete labels on 2D connectivity graphs with edge masks. Neither handles continuous energies, 3D geometry, or the question of how many atoms jointly drive an output, so they serve as motivation rather than direct competitors.
On the physics side, Quadruple Attention in Many-body Systems (MABNet) builds explicit four-atom terms and reports about 20% lower MAE on MD22 and 67% better forces than TensorNet on SPICE dipeptides. That is evidence that explicit order matters, but it says nothing about what order a trained model effectively uses. EGraFFBench shows that the lowest-static-error models (TorchMDNet, Equiformer) often explode in simulation, while MACE, NequIP and Allegro preserve structure. It offers no mechanism, and our third experiment targets that gap. Using graph neural network and symbolic regression to model disordered systems recovers $3.958(1-x^6)/x^{12}$ from a pairwise network on low-temperature Lennard-Jones data. The same pipeline fails at high temperature because passive data under-samples the repulsive wall. This supports our choice of active probes over passive fitting, and supplies a pairwise ground-truth system. Fast general two- and three-body interatomic potential (GTTP) is a fixed-order spline potential and serves as a strong fixed-order competitor.
Research Program
We will build a probing library that computes $I_S$ for sets of size two through four. It will use both finite differences at controlled $\delta$ and nested automatic differentiation, which needs only Hessian-vector-style products, so cost grows with the number of sampled sets rather than with all $O(N^4)$ quartets. Sets are sampled within the model cutoff, stratified by geometry, separating bonded triplets from non-bonded ones. Directions are random unit vectors, which avoids hand-picking. The summary statistic for order $k$ is the spurious-order mass $M_k$: the average squared $I_S$ over sets of size $k$, normalized by the average at order two.
We then apply the library to three settings. First, systems with known ground truth, where every answer is checkable. Second, the hydrogen octamer, where the paradox was found. Third, the EGraFFBench molecules and materials, where simulation stability is the outcome. Throughout, the deletion-based decomposition is computed on the same models as the main comparator.
Baselines
The naive reference point is the architecture's nominal body order combined with static force MAE: assume SchNet-like models are effectively two-body, MACE matches its correlation order, and PET is unbounded, and rank stability by MAE. This is what practitioners do by default. EGraFFBench already shows that static MAE fails as a stability predictor, but the file gives no single reported scalar for that comparison, so no reported result found on Lacuna for an EGraFFBench stability-versus-MAE correlation coefficient. We will compute it ourselves on identical rollouts.
The strongest existing method for the body-order question is the deletion-based many-body expansion of the body-order paradox paper. The model is evaluated on every sub-cluster and the energy is decomposed by inclusion–exclusion over atom removal. Its reported operating point is under 0.025 eV/atom RMSE on full octamers, with a 3.4-fold error increase for MACE when sub-clusters are enforced in training. We will run it on exactly the same trained checkpoints, configurations and atom sets as our probe, so the only difference is the intervention: removing versus nudging atoms. The displacement probe beats it if, on systems with known body order, it recovers the true zero pattern and nonzero magnitudes while the deletion probe reports spurious higher-order terms. Concretely, we would need lower relative error of the three-body contrast against ground truth on held-out configurations, and spurious four-body mass at least an order of magnitude closer to zero.
The associational competitor is a gradient-times-input or GNNExplainer-style edge attribution adapted to energy regression, run with the same compute budget per configuration. It tests whether ordinary attribution already tells us which atom groups matter. CXGNN's motif results suggest it will not, but it must be measured here.
The key ablation sweeps the amplitude $\delta$ from thermal scale up to displacements comparable to bond lengths. This deliberately moves the probe off-manifold. If the hallucinated decomposition reappears only as $\delta$ grows, the on-manifold ingredient is what matters. If it appears at all $\delta$, our reinterpretation of the paradox is wrong.
Why not just…?
Why not just use the sub-cluster deletion analysis that already exists? Because whether it is faithful is precisely what is in doubt. Deleting atoms creates fragments outside the training distribution. A model can be exactly physically decomposed on real configurations and still return arbitrary fragment energies. The known-ground-truth experiments settle this directly. If deletion and displacement agree on synthetic Lennard-Jones and three-body silicon models trained to comparable accuracy, deletion was fine and our contribution narrows to a cheaper probe. We would report that.
Why not just compare the model's Hessian, or its forces, to reference calculations? Force and Hessian errors measure how wrong the model is, not which order of interaction it uses. A Hessian couples only two atoms at a time and mixes contributions from all body orders into each entry. The three- and four-atom contrasts are the lowest-order quantities that vanish exactly for lower-order potentials. The comparison is still informative, so the stability experiment includes Hessian error as a competing predictor. If it predicts rollout failure as well as spurious-order mass, our diagnostic adds nothing for stability, and we will say so.
Why not just use an attribution method or a causal graph explainer such as CXGNN? These assign importance to atoms or edges, which says which atoms matter but not whether they act jointly or additively. CXGNN also trains one causal model per candidate neighborhood, which does not scale to trajectories. We include an attribution baseline. Its failure mode would be visible as high importance scores on atoms whose joint contrast is zero.
Why not just train fixed-order potentials such as GTTP or MACE at a set correlation order, and skip diagnosis? This is a real threat for the stability question. If fixed-order models are both accurate and stable, explaining unconstrained ones matters less. MABNet's gains from explicit four-body terms, and the paradox paper's evidence that unconstrained PET fits octamers well, suggest flexible models stay valuable. Our final experiment therefore asks a narrower question: does penalizing on-manifold spurious-order mass give a flexible model the stability of fixed-order models without losing their accuracy? If it does not, the practical recommendation is simply to use fixed-order potentials.
Proposed Experiments
The first experiment uses ground truth we generate. We simulate the 128-atom disordered Lennard-Jones system at temperatures below and above the glass transition, following the Chen et al. setup. We add a Stillinger-Weber-style silicon system with known two- and three-body terms, and a small molecule with an added analytic torsion term giving known four-body structure. We then train SchNet-like, NequIP, MACE and a PET-style transformer to matched force error. For each model, we compute displacement contrasts and deletion contrasts against the analytic truth. The decisive check is whether trained models show near-zero three-atom contrast on Lennard-Jones and near-zero four-atom contrast on silicon under displacement, while deletion reports nonzero terms. The high-temperature Lennard-Jones split tests whether active nudging recovers the repulsive regime where passive symbolic distillation failed.
The second experiment revisits the hydrogen octamer dataset. We retrain SOAP-BPNN, MACE and PET to the reported under-0.025 eV/atom level and compute displacement contrasts. We compare these to reference contrasts obtained by finite-differencing reference forces, which is feasible for eight atoms. The paradox is reinterpreted if displacement contrasts match the reference while deletion decompositions diverge. We will also test whether MACE trained with sub-clusters, the 3.4-fold-worse model, is closer to the reference under deletion but not under displacement. That pattern would show the penalty comes from fitting fragments, not from physical decomposition per se.
The third experiment links mechanism to stability. On 3BPA, MD17, GeTe and LiPS20, using the EGraFFBench protocol and its structural metrics (Jensen-Shannon divergence of radial distributions and Wright's factor), we train each architecture several times. We then ask whether spurious-order mass, measured on configurations slightly hotter than training, predicts rollout failure better than static force MAE and Hessian error. The causal test follows. We retrain with a penalty on spurious-order mass above order four, sampled on such configurations, and hold static MAE fixed within tolerance. If stability improves at equal static error, high-order response is a cause of instability rather than a correlate. If stability does not move, the diagnostic is descriptive only, and the result narrows the proposal to the interpretability question of the first two experiments.
References
- Resolving the body-order paradox of machine learning interatomic potentials
- Three Desiderata for Faithfulness in Machine Learning Explanations: The Case for Causal Abstraction
- The explanation game: a formal framework for interpretable machine learning
- Conceptual challenges for interpretable machine learning
- Graph Neural Network Causal Explanation via Neural Causal Models
- OrphicX: A Causality-Inspired Latent Variable Model for Interpreting Graph Neural Networks
- Quadruple Attention in Many-body Systems for Accurate Molecular Property Predictions
- EGraFFBench: Evaluation of Equivariant Graph Neural Network Force Fields for Atomistic Simulations
- Using graph neural network and symbolic regression to model disordered systems
- Fast general two- and three-body interatomic potential