Label-Free Adversarial Uncertainty Training for Neural Interatomic Potentials

Neural networks trained to imitate density functional theory (DFT, the standard quantum-mechanical method for computing energies and forces on atoms) now replace DFT inside molecular dynamics and catalyst screening. They fail silently. When a simulation wanders into an atomic arrangement unlike anything in the training data, such as a compressed bond or an unvisited transition state, the network still returns confident forces, and the trajectory can blow up. The reliable fix today is a deep ensemble, five to ten independently trained networks whose disagreement flags trouble. That multiplies cost on datasets where a single model already needs thousands of GPU days.

We propose to teach a single model where it should be uncertain, without paying DFT to label the places it does not know. The idea comes from out-of-distribution entropy maximization in image classification. There, a network is shown synthetic or adversarially chosen inputs and trained to output maximal uncertainty on them, with no labels required. We transfer this to atomistic regression in two steps. First, during training, we search atomic coordinates for displaced geometries where the model's predicted force variance is suspiciously small. Second, we penalize the model whenever that variance falls below a floor that grows with distance from the training data. If this works, one model with a variance head would match ensembles at catching force errors and preventing simulation crashes, at roughly ensemble-of-one cost.

Core Claim

Single-model uncertainty for interatomic potentials fails out of distribution because nothing in training ever tells it what "far from data" means, not because single models are intrinsically incapable. The comparison study Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles found three weaknesses in PaiNN models. Mean-variance estimation, which adds a head predicting a Gaussian variance, was "the least capable" at flagging out-of-distribution errors on ammonia inversion. Deep evidential regression was numerically unstable. Ensembles won. Every single-model method in that study saw only labeled in-distribution data. We claim that an adversarial, label-free variance floor closes most of the gap to ensembles on out-of-distribution error detection and molecular dynamics stability, and does so without measurably raising in-distribution force error.

The training objective adds one term to the usual Gaussian negative log-likelihood $\mathcal{L}_{\text{NLL}}$ on DFT-labeled configurations:

$$\mathcal{L} = \mathcal{L}_{\text{NLL}} + \lambda \, \frac{1}{N}\sum_{i=1}^{N} \Big[\max\big(0,\; \log s(\delta_i) - \log \hat{\sigma}_i(\tilde{R})\big)\Big]^2 .$$

Here $\tilde{R}$ is an adversarially displaced copy of a training geometry with $N$ atoms. The quantity $\hat{\sigma}_i(\tilde{R})$ is the model's predicted per-atom force standard deviation, and $\delta_i$ is atom $i$'s displacement from the nearest training geometry in the batch. The variance floor is $s(\delta) = \sigma_0 + \kappa\,\delta$, and $\lambda$ weights the penalty.

The floor is graded rather than flat on purpose. Classification-style entropy maximization pushes outliers to a uniform distribution, which is known to wash out legitimate confidence near decision boundaries. In regression, a small displacement should only cost a small amount of confidence. We set $\kappa$ from how force error actually grows with displacement on a small labeled validation set of rattled structures, and OC20 contains such structures. A linear form is the first-order Taylor behavior of force error under small perturbations. The adversarial geometry $\tilde{R}$ is found by a few projected gradient steps that minimize $\hat{\sigma}$, mirroring how ACET searches for the worst-case overconfident input. The search is restricted to a displacement shell $[r_{\min}, r_{\max}]$ and rejects geometries whose interatomic distances fall below a fixed fraction of covalent radii. The restriction keeps the search away from nuclear collapse, where the floor is trivially satisfied.

Relation to Prior Work

Two papers supply the method. Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem shows that ReLU networks become arbitrarily confident far from data. It introduces ACET, adversarial low-confidence training on noise, which cut mean maximal confidence on noise from 99.8% to 10.0% on MNIST without hurting CIFAR-10 accuracy. Reliable Uncertainty Estimates in Deep Neural Networks using Noise Contrastive Priors defines priors in input space by perturbing batch inputs and asking for a wide output distribution there. We borrow ACET's inner adversarial search and NCP's idea of perturbing the current batch. We replace both of their flat maximum-uncertainty targets with a displacement-graded floor suited to continuous forces, and we work in $3N$-dimensional coordinate space, where the ReLU polytope argument does not directly apply to smooth activations.

Three atomistic papers motivate the gap and also set its limits. Adversarial Attacks on Uncertainty Enable Active Learning for Neural Network Potentials ascends ensemble force variance to find informative geometries, then labels every one with DFT. When Forces Disagree: A Data-Free Fast Uncertainty Estimate for Direct-Force Pre-trained Neural Network Potentials attacks the disagreement between direct and gradient-derived forces, but only to evaluate models after training. Evidential deep learning for interatomic potentials builds per-atom, per-Cartesian-component evidential heads on PaiNN and uses uncertainty-gradient dynamics to propose new DFT points. Its heads are still fit only on labeled data.

Our proposal differs from all three in one way: adversarial geometries are never sent to DFT and never used only for evaluation. They become unlabeled training targets for the uncertainty head. We adopt eIP's locality and directionality as architectural choices, so per-atom and per-component variances, rather than as the contribution.

Uncertainty quantification in molecular simulations with dropout neural network potentials is a constraint. Its authors concede that dropout uncertainty may underestimate error under extreme extrapolation. That is the failure we target.

Research Program

We build the regularizer as a drop-in training wrapper for two backbones. PaiNN serves the molecular benchmarks, where the prior comparisons were run. A PhAST-accelerated GemNet-OC, following PhAST: Physics-Aware, Scalable, and Task-Specific GNNs for Accelerated Catalyst Design and GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets, serves the catalytic surfaces. We attach the regularizer to two uncertainty heads, a plain mean-variance head and an eIP-style evidential head. This lets us answer the open question in Gap 3: is evidential instability caused by the lack of an out-of-distribution anchor? If the regularized mean-variance head matches the regularized evidential head, we will recommend the simpler one.

Two analyses accompany the method. The first measures how the inner adversarial search scales with the number of attack steps and with molecule size. It decides whether a fixed small budget, around five gradient steps on a quarter of each batch, suffices. The second compares two choices of floor. One is the graded floor. The other replaces it with $\max(s(\delta), U_\Delta)$, where $U_\Delta$ is the Force Delta from the When Forces Disagree paper. That second choice is possible only on direct-force models, and tests whether a data-free physical disagreement signal makes a better target than a geometric prior.

Baselines

The naive reference point is a plain mean-variance PaiNN trained on labeled data only, exactly as in the single-model comparison study. Its known out-of-distribution failure is what we claim to repair. Beating it is necessary but uninteresting.

The strongest competitor is a deep ensemble of five PaiNN models, the method the comparison study found most reliable at catching failures and preventing crashes. We give it every advantage on accuracy: same data, same per-member hyperparameter search, five times the training compute. We then report our method both at equal model count and at equal GPU-hours, where our single model trains longer. Beating the ensemble means matching or exceeding its Spearman correlation between predicted uncertainty and true force error, and its trajectory stability, on the out-of-distribution sets, at one fifth of the inference cost. Matching it within noise at that cost counts as success. The comparison paper's text gives no ensemble Spearman or AUC values on ammonia inversion (no reported result found on Lacuna for that number), so we rerun the ensemble ourselves.

The strongest single-model competitor is eIP from Evidential deep learning for interatomic potentials, which reports Spearman correlations between uncertainty and force error "up to 0.86" on ISO17. We reproduce it on our splits with its published design and give it the same tuning budget.

On direct-force models the comparison is post-hoc Force Delta, which reports Spearman correlations "up to 0.98" on equilibrium structures and beat a 10-model ensemble on EquiformerV2. Because that figure is for equilibrium structures, we evaluate on adversarial and transition-state geometries, where the method has not been shown to hold.

For data efficiency the reference is labeled adversarial active learning, which reports an ammonia inversion barrier within 0.25 kcal/mol of DFT by generation two and 83% versus 63% alanine dipeptide trajectory stability against random sampling. We treat it as an upper bound that spends DFT labels, and we report how many labels it needs to beat our label-free model.

The key ablation swaps the adversarial search for isotropic Gaussian displacements of the same magnitudes, which is NCP/CEDA-style training with the same graded floor. This isolates whether the adversarial search, rather than the floor alone, carries the effect. A second ablation uses a flat floor in place of the graded one, which isolates the accuracy tax.

Why not just…?

Why not just use a deep ensemble? It is the real threat, and the comparison study says it works. Our argument is cost, not superiority. On OC20 the GemNet-OC authors cite full-dataset training "exceeding 16,000 GPU days" for one model, and ensembles multiply inference cost at every molecular dynamics step. Ensembles also have a documented weakness: members sharing an inductive bias can agree confidently and wrongly. Our adversarial evaluation set, described below, is built to expose exactly this. If ensembles still dominate at equal GPU-hours, we will report that single-model regularization falls short.

Why not just label the adversarial geometries with DFT, as active learning does? When DFT is cheap relative to the system, it should. Labels improve the energy surface, while our method only improves the warning signal. But each active-learning round pays one DFT calculation per geometry, and most adversarial geometries are useless high-energy states. Our experiments measure the trade directly. We count how many DFT-labeled adversarial points the active-learning baseline needs before its uncertainty detection overtakes our label-free model. We also test the two together: our regularizer as the uncertainty source for active learning, replacing the ensemble.

Why not just add random coordinate noise, which is cheaper than an adversarial search? The active-learning paper found random distortions consistently overestimated the ammonia barrier and gave 63% versus 83% trajectory stability. ACET's authors likewise found noise training leaves confident pockets. Both results concern data selection or classification, though, not variance-floor regularization, so the isotropic ablation above is run at equal compute. If it matches the adversarial version, the simpler noise method is the contribution and we will say so.

Why not just fit a density model, such as a Gaussian mixture, on frozen latent features? The comparison study found Gaussian mixtures competitive for detection, but they need manual tuning of the number of clusters. They also give one scalar per atom rather than per-component force uncertainty. We include them as a post-hoc baseline on identical backbones.

Proposed Experiments

The molecular study uses PaiNN on rMD17 for interpolation, the ammonia inversion benchmark, bulk silica glass, and ISO17. We measure in-distribution force mean absolute error to quantify the accuracy tax, along with Gaussian negative log-likelihood. Out of distribution, we measure Spearman correlation between predicted uncertainty and true force error, and the area under the ROC curve for flagging atoms whose force error exceeds a fixed threshold. We also measure alanine dipeptide trajectory stability under the protocol of the active-learning paper, so that its 83% figure is a direct reference.

The catalytic study trains on OC-2M and evaluates on the OC20 S2EF out-of-domain splits for unseen adsorbates and catalysts, plus relaxation trajectories where predicted uncertainty should rise before relaxations diverge. The direct-force study uses the ten-system Force Delta benchmark, CaPd-NH2 among them.

The decisive check is a held-out adversarial test set. For every trained model, including the ensemble and the uncertainty heads, we run an attack by a different model's procedure. One example is the Force Delta composite attack, which lowers energy while raising disagreement. We label a few hundred resulting geometries per system with DFT and ask whose uncertainty tracks the true error. Evaluating each model against an attack it was not trained against guards against the method simply learning to defeat its own attacker. The proposal succeeds if the regularized single model meets three conditions on these held-out attacks and on ammonia inversion: it matches the five-member ensemble in Spearman correlation and error-flagging AUROC, it raises in-distribution force error by under 5%, and it beats the isotropic-noise ablation. It fails cleanly if the isotropic ablation does as well, which would mean the adversarial search adds nothing. It also fails cleanly if the ensemble's lead persists at equal GPU-hours.