Choosing How Much a Molecular Model Should See: Expressivity Matched to Size Transfer in Electron-Density Prediction

Machine-learned chemistry models are usually trained on small molecules because accurate quantum calculations on large ones are too expensive. They are then expected to work on large molecules. This works when each atom's behaviour is set by its neighbourhood, because a large molecule is then mostly built from neighbourhoods already seen in training. It breaks in two opposite ways. If the model's view of a neighbourhood is too coarse, atoms that behave differently look identical to it, and no amount of training separates them. Graph neural networks have exactly this blind spot, bounded by a classic graph-matching test called 1-WL that cannot tell a hexagon from two triangles. If the model's view is too fine, large molecules contain neighbourhoods it has never seen, and its predictions on them are arbitrary. This proposal argues that the right amount of structural detail can be chosen before testing, from training data alone. It then tests that claim on a real size-transfer problem: predicting electron densities that warm-start quantum chemistry calculations on molecules three times larger than any in training.

Stated for a colleague: every way of encoding an atom's surroundings sorts atoms into "look-alike" classes. Size transfer succeeds when those classes are both pure (look-alikes really have similar targets) and covered (classes found in large molecules already appear in small ones). We propose to measure purity and coverage on the training set, use them to pick the encoding level, and show that this choice predicts which model transfers best to large molecules.

Why this question, and why now

Towards a Transferable Acceleration Method for Density Functional Theory introduced SCFbench. It has 43,862 training molecules of at most 20 atoms and 1,050 test molecules of 26–60 atoms. The paper showed that predicting the local electron density with an equivariant network transfers across this size gap. Its NequIP-L model, a rotation-aware message-passing network, keeps the Relative Iteration Count (RIC), the ratio of self-consistent-field iterations from the ML initial guess to iterations from the standard guess, at about 66% across sizes. Hamiltonian-matrix models collapse to RIC values as high as 179%. The same paper reports that the ground-truth density would allow roughly a 60% reduction. The best transferable model therefore recovers about half of the available speedup. That model uses only spherical distance cutoffs. It has no notion of which ring an atom belongs to, or of conjugation paths that leave its cutoff sphere.

The obvious move is to add ring topology. Work escaping the 1-WL limit already does this on 2D graphs. Weisfeiler and Lehman Go Cellular: CW Networks lifts rings to "2-cells" that pass messages. Expressivity and Generalization: Fragment-Biases for Molecular GNNs gives a hierarchy of ways to attach ring and path fragments: as node features, as fragment nodes, or as a higher-level fragment graph. E(n) Equivariant Message Passing Cellular Networks brings cells into 3D equivariant models, but it tests only on QM9-sized molecules and uses a dense all-pairs geometric track. None of these tests transfer from small to large molecules, and none predicts electronic fields.

Two results warn against simply adding as much structure as possible. From Local Structures to Size Generalization in Graph Neural Networks proves that unseen local patterns in large test graphs allow zero-training-error solutions with arbitrary test error. Larger receptive fields make such patterns more likely. Graph Representational Learning: When Does More Expressivity Hurt Generalization? proves and shows that expressivity beyond what the task's invariants require hurts generalization. It leaves open how to find those invariants in real chemistry. Our proposal turns that open question into a measurement.

Core Claim

Fix an equivalence relation $R$ on atom environments, meaning a rule that declares two atoms' surroundings "the same." Examples are 1-WL color after $d$ rounds, the same color plus the sizes and aromaticity of rings the atom lies in, or the same plus the fragment-graph neighbourhood. A local model limited to $R$ must give every atom in a class the same prediction, up to the continuous geometry it also sees. Its per-atom error on large molecules should then decompose roughly as

$$ \mathbb{E}[\text{err}_R] \approx (1-c_R)\, e_{\text{unseen}} + c_R\, \sigma^2_R , $$

where $c_R$ is the fraction of test atoms whose $R$-class appears in training, $e_{\text{unseen}}$ is the typical error on never-seen classes, and $\sigma^2_R$ is the within-class variance of the target density coefficients after regressing out geometry. The form comes from splitting test atoms into covered and uncovered sets. Covered atoms pay the irreducible within-class spread; this is the function-approximation consequence of indistinguishability shown in On the equivalence between graph isomorphism testing and function approximation with GNNs. Uncovered atoms pay the arbitrary-extrapolation cost described by Yehudai et al.

Refining $R$ lowers $\sigma^2_R$ and lowers $c_R$. The best encoding sits where the two balance. Both quantities can be estimated inside the training set by treating its largest molecules as stand-ins for the large test molecules.

The new element is not ring features themselves. It is a training-data-only criterion that predicts which level of the expressivity ladder transfers best across molecular size, validated on an electronic-structure target where the answer has a direct cost in solver iterations.

Research Program

Build the ladder. All rungs share the NequIP-L backbone and its cutoff, so each rung differs only in the discrete structural information added. They form a sequence of increasingly fine equivalence relations.

  • The base rung is the original model.
  • The next rung adds ring membership as atom features: size, aromaticity, and fused-ring count. This is the cheapest level in the Fragment-WL hierarchy.
  • The next adds fragment nodes connected to their atoms, using FragNet's rings-and-paths decomposition with ordinal size encoding, so that unseen ring sizes still map somewhere sensible.
  • The top rung adds a higher-level fragment graph in which fragments exchange messages along shared bonds and conjugated paths. Information can then travel along a conjugated system beyond the geometric cutoff.

Fragment messages are restricted to adjacent fragments. This keeps cost linear in molecule size, unlike the quadratic attention of Higher-Order Molecular Learning: The Cellular Transformer or EMPCN's dense track. Aggregation over atoms stays a sum because density is extensive. We do not adopt the mean aggregation that On Transferring Transferability: Towards a Theory for Size Generalization recommends for limits under node duplication. That limit models growing graphs with fixed local statistics, whereas molecules grow by adding atoms with bounded neighbourhoods. We treat this as an assumption to check, not as settled.

Measure purity and coverage. For each rung's relation, we compute three quantities.

  • Classes are the discrete part of the environment: element, bond-graph pattern of radius $d$, and the rung's ring or fragment labels.
  • Purity $\sigma^2_R$ is the within-class variance of reference density coefficients, after a small geometry-only regressor removes conformational spread.
  • Coverage $c_R$ is computed by splitting training molecules at 14 atoms and asking what fraction of atoms in 15–20-atom molecules fall into classes seen among smaller ones.

Combining these with an $e_{\text{unseen}}$ estimated on the same pseudo-test split gives a predicted transfer error for each rung. We compute the same quantities on the true 26–60-atom test set only after the predictions are fixed.

Analyze where the gap to ground truth comes from. We stratify per-atom density errors on the large molecules by covered versus uncovered status and by ring context (non-ring, isolated ring, fused or conjugated system). This tells us whether the remaining distance to the ground-truth bound is mainly a purity problem that more structure can fix, or a coverage problem that only more diverse small-molecule data can fix.

Baselines

The naive reference point is the standard DFT initial guess, which defines RIC = 100%. Alongside it we report the Hamiltonian-prediction route in SCFbench, which reaches RIC as high as 179% on large molecules. These anchor the scale.

The strongest existing method is NequIP-L predicting electron density, as reported in Towards a Transferable Acceleration Method for Density Functional Theory, with RIC ≈ 66% from ≤20-atom training to 26–60-atom testing. It is the strongest competitor because it is the only method in the file shown to hold its gain across this size gap. We retrain it ourselves on identical splits, auxiliary basis, functional, parameter count, and hyperparameter search budget, then confirm we reproduce roughly 66% before comparing. Beating it means a lower RIC on the out-of-distribution set, with a gap larger than the spread across three random seeds. Because the ground-truth density bound implies roughly 40% RIC, we also report the fraction of the remaining gap each rung closes.

Two baselines represent "more context without topology": a larger geometric cutoff, and two extra message-passing layers. Both are tuned under the same compute budget.

One baseline represents "global structure without locality": Laplacian positional encodings, which Benchmarking Graph Neural Networks shows break the 1-WL ceiling on synthetic tasks, moving circular skip link accuracy from 10% to 100%.

An EMPCN-style model, the closest prior architecture, is run at matched parameters on the same splits.

Pattern-tree self-supervised pretraining from Yehudai et al. targets coverage rather than purity, so we include it as a competitor on the coverage side of the decomposition.

The ablation that isolates the proposal's key ingredient compares two ways of choosing a rung. One uses our purity–coverage criterion. The other picks the rung with the best in-distribution validation error, which is what practitioners do by default. The proposal succeeds only if our criterion picks a rung at least as good out of distribution, and better whenever the two choices disagree.

Why not just…?

Why not just add ring-membership features to NequIP-L? This is the most serious threat. Ring size and aromaticity are cheap and may capture most of the purity gain. We do not argue around it. It is the second rung of the ladder and is evaluated exactly like the others. If it matches the fragment-graph rungs, the paper's result becomes "the minimal ring-aware level suffices for density transfer." The test is whether the purity–coverage criterion predicted that outcome from training data. The criterion, not the ring encoding, is the claim at stake.

Why not just enlarge the cutoff or add layers? Yehudai et al. prove that larger receptive fields increase the rate of unseen patterns in big graphs. Larger cutoffs also raise cost per atom. The cutoff baseline measures this directly. In our framework a larger cutoff is itself a finer relation, and should show high purity but falling coverage. If it nonetheless wins, that is evidence against the coverage term mattering for this target.

Why not just use an existing cellular equivariant model such as EMPCN? It has never been tested on size transfer or on density targets. Its dense all-pairs geometric track grows quadratically, which conflicts with the linear scaling that makes density prediction attractive. We run it anyway at matched budget; if it transfers well, our ladder's top rung is unnecessary.

Why not just use Laplacian positional encodings? Their eigenvectors depend on the whole graph and change as molecules grow, so they are unlikely to be covered across sizes. This concern is consistent with Levin et al.'s analysis of which operators are continuous across sizes. The Laplacian baseline tests it. We would expect good in-distribution error and weak transfer, and the criterion should flag it as low coverage.

Proposed Experiments

The main experiment trains every rung and baseline on SCFbench's ≤20-atom set and reports RIC and per-atom density-coefficient error on the 1,050 molecules of 26–60 atoms. We include the PBE-trained, B3LYP-tested cross-functional setting the original paper reports, to check that structural gains survive a change of functional.

The decisive check is the rank correlation between predicted and observed out-of-distribution error across rungs. Predictions are frozen before the large test molecules are touched. A positive correlation that beats in-distribution validation error as a selector supports the claim. A null correlation refutes it, regardless of which rung happens to win.

Two secondary checks follow. First, a controlled stress test in the style of FragNet's 7-membered ring experiment: we remove all molecules with fused or 7-plus-membered rings from training and verify that coverage collapses as predicted on rungs that encode those rings explicitly, and that ordinal ring-size encoding restores it. Second, the stratified error analysis described in the research program. It reports whether the half of the ground-truth speedup that NequIP-L misses lies in uncovered environments or in impure covered ones. That answer decides whether the next step for transferable DFT acceleration is richer structure or broader small-molecule data.