Neural networks are now routinely used to correct nuclear mass models: a physics model predicts how tightly each nucleus is bound, and a small network learns the leftover error from measured masses. The prize is extrapolation. These corrections are trained on roughly two and a half thousand measured nuclei, but they are used to predict exotic, neutron-rich nuclei that have never been measured. Such predictions determine where the neutron drip line lies (the edge of the nuclear chart, past which a nucleus cannot hold another pair of neutrons) and feed into models of how heavy elements form in stellar explosions. The network architecture behind each published correction, meaning its depth, width, activation and input encoding, is chosen by hand or by random-split validation error. That criterion rewards filling gaps inside the measured region. It says nothing about behaviour at the edge.
We propose to measure, for the first time, how much the predicted drip line moves when one swaps between architectures that look equally good on standard validation. We also ask whether any cheap selection signal predicts which architectures extrapolate well. The tool comes from neural architecture search (NAS): a tabular benchmark, a precomputed table of thousands of fully trained architectures that turns questions about architecture selection into cheap lookups. In NAS the expensive quantity is training. In nuclear masses training is cheap, and the expensive quantity is ground truth beyond the measured region, which arrives only when new isotopes are measured. Our methodological move is to treat "extrapolation quality" the way NAS treats "fully trained accuracy": a target that must be predicted by a cheap proxy, with the proxy's reliability checked on held-out data.
Core Claim
Among mass-correction networks with indistinguishable random-split error, predicted two-neutron drip lines can differ by amounts comparable to the disagreement between competing nuclear density functionals. Random-split validation error is a poor ranker of extrapolation quality, especially among the best architectures. A validation protocol that holds out the most neutron-rich measured isotopes of each element ranks extrapolators substantially better at no extra cost.
The drip-line quantity we track is the two-neutron separation energy,
$$S_{2n}(Z,N) = B(Z,N) - B(Z,N-2),$$
where $B(Z,N)$ is the binding energy of a nucleus with $Z$ protons and $N$ neutrons. The predicted two-neutron drip line for element $Z$ is the largest $N$ for which $S_{2n}(Z,N) > 0$. The reference scale for "comparable" is concrete. In Studies of Deformed Halo Structures of 39Na and 42Mg, coordinate-space Skyrme calculations with the SkM*ext1 functional end the magnesium chain at ^42Mg, while UNEDF0ext1 extends it to ^44Mg. That is a one-neutron-pair disagreement that the field treats as a genuine physics uncertainty. If architecture choice alone produces shifts of that size, every network-corrected drip-line prediction carries an unreported error bar.
Prior Work and Where This Sits
Two papers supply the method we are auditing. Deep learning for nuclear masses in deformed relativistic Hartree-Bogoliubov theory in continuum trains a hand-chosen multilayer perceptron with ELU activations on four physics features: proton number, neutron number, a pairing term, and a valence-nucleon shell term. It fills gaps in a deformed mean-field mass table and reports 0.842 MeV RMS deviation overall and 0.725 MeV on newly measured neutron-rich isotopes. Machine learning the nuclear mass instead learns residuals of four classic mass models with gradient-boosted trees (LightGBM). On AME2016 it reduces the WS4 model's error from about 0.302 to about 0.170 MeV, and it notes a clear degradation for light nuclei. Neither paper searches over architectures. Neither asks whether its chosen model's drip line is stable under that choice. We do not propose a better mass model. We propose to measure the variance these papers leave unquantified.
The NAS literature supplies the methodology and constrains our ambitions. NAS-Bench-201 and NAS-Bench-101 showed that exhaustively training a search space once makes selection research reproducible and cheap. NAS-Bench-201 also found that random search matches elaborate search algorithms. We take that negative result seriously, so we propose no search algorithm and focus only on the selection signal.
Zero-Cost Proxies for Lightweight NAS and EZNAS show that scores computed at initialization, from a single forward or backward pass, can rank image classifiers. These scores include gradient norms, saliency metrics such as SNIP and Synflow, and evolved expressions. EZNAS also documents that these scores lose discrimination among the top decile of architectures, and both papers validate only on vision and language networks. On Random Weights and Unsupervised Feature Learning motivates the hope that an architecture's inductive bias is visible before training. We test that hope where it matters for physics, on the smoothness of the untrained function along isotopic chains. Rethinking Performance Estimation in Neural Architecture Search supplies the budgeted-training idea we use to fix a fair, minimal training length per architecture.
Research Program
We will build a nuclear mass architecture benchmark. The search space covers residual-correction networks on top of each of four base models: liquid drop, Duflo-Zucker, WS4 and FRDM. It varies depth from one to six layers, width over a geometric grid, activation (ELU, tanh, ReLU, SiLU), the input feature set (raw $Z,N$; plus pairing and shell terms as in Choi et al.; plus distance-to-magic-number as in the LightGBM work), optional Fourier-feature embeddings of $(Z,N)$, and weight decay. We expect several thousand configurations. Each will be trained with five seeds on AME2016. Each run fits a few thousand points and finishes in well under a minute on a CPU, so the whole table is a modest cluster job. This cheapness is exactly why we do not need zero-cost proxies to save compute.
Every trained model is evaluated on three held-out sets. The first is a random 10% split, the standard practice. The second is an edge split that removes the two most neutron-rich measured isotopes of every element from training. The third is a time split: the 66 nuclei newly measured in AME2020 and used for validation in the LightGBM paper. These are disproportionately neutron-rich and are the only true out-of-sample extrapolation test available. For every model we also record the predicted $S_{2n}$ along each isotopic chain and the implied drip-line position. Light nuclei with $A<16$, where alpha-clustering such as the ^10Be molecular structure confirmed in Validation of the 10Be Ground-State Molecular Structure defeats smooth models, are reported separately so they do not dominate the conclusions.
On this table we will answer three questions. First, the spread question: among architectures within one seed-standard-deviation of the best random-split error, how far apart are their predicted drip lines, per element and summed over the chart, and is that spread larger than seed-to-seed spread within one architecture? Second, the ranking question: how well do random-split error, edge-split error, and initialization-time scores rank the architectures on time-split error, measured by Kendall's tau overall and within the top decile? Third, the transfer question: does the edge-split criterion selected on AME2016 choose architectures that also do well on the AME2020 additions, which is the only test that cannot be gamed?
Baselines
The naive reference point is the base physics model with no correction. For WS4 this is about 0.302 MeV RMSD on AME2016, as reported in Machine learning the nuclear mass. It sets the floor any correction must clear on the time split, where corrections can plausibly hurt.
The strongest existing correction is LightGBM residual learning on WS4, with about 0.170 MeV RMSD on AME2016 in that same paper. We will rerun it on our exact splits, with its ten published features, and give it a hyperparameter search budget equal in trained-model count to the neural selection procedure. For extrapolation specifically, we found no reported RMS number on Lacuna for LightGBM on the 66 AME2020 nuclei; the paper states only that generalization was stable. We will therefore measure it ourselves. Trees predict piecewise constants outside their training range, so we expect them to be competitive on the random split and weaker on the edge and time splits. Showing this directly is part of the contribution. Our selected network beats this baseline only if its time-split RMS is lower and its drip-line predictions remain stable across seeds.
The neural reference is the Choi et al. architecture: an ELU multilayer perceptron with $Z$, $N$, pairing and shell inputs. It reported 0.842 MeV overall and 0.725 MeV on new neutron-rich isotopes against DRHBc-plus-experiment data in Deep learning for nuclear masses in deformed relativistic Hartree-Bogoliubov theory in continuum. Those numbers are on a different base model and are not directly comparable. We will instead place this architecture inside our benchmark on the WS4 residual task and report its rank under each criterion. This tells us whether a sensible human choice already lands near the extrapolation-optimal region.
The key ablation isolates the selection criterion and holds the search fixed. Random search draws the same number of architectures, from the same space, with the same training budget, and selects either by random-split error or by edge-split error. Any difference in time-split error and drip-line stability is then attributable to the criterion alone. A second ablation, the zero-cost route, replaces trained validation with the Synflow, SNIP and gradient-norm scores from Zero-Cost Proxies for Lightweight NAS, plus a physics-specific initialization score: the mean squared second difference of the untrained network's output along $N$ at fixed $Z$. On NAS-Bench-201, the best reported zero-cost ranking was a vote of three proxies reaching 0.82 Spearman correlation with trained accuracy. We will report the analogous correlation against time-split error, with no expectation that the vision result transfers.
Why not just…?
Why not just pick architectures by ordinary random-split validation, as everyone does? This is the hypothesis under test, not a dismissed alternative. If random-split error ranks time-split error as well as edge-split error does, including in the top decile, our selection claim fails. We would report that standard practice is adequate. The spread measurement would remain informative either way.
Why not just use LightGBM, which is cheaper and already more accurate in-sample? On the random split it may well win; its reported 0.170 MeV on WS4 residuals is strong. But extrapolation is the use case, and tree ensembles cannot produce a trend beyond the training range, which matters directly for $S_{2n}$ falling toward zero. Its own authors also document degradation on light nuclei. The time split settles this. If LightGBM matches the best network on the AME2020 additions and gives stable drip lines, we will say so, and the architecture question becomes moot for practitioners.
Why not just average over many architectures and report the ensemble spread as uncertainty? This is a real threat. It may be the right practical answer, and the benchmark makes it trivially available. We will include an ensemble over randomly drawn architectures, and an ensemble over the top-k under each criterion, as competitors. What averaging cannot tell us is whether the spread is honest. If all top random-split architectures share the same extrapolation bias, the ensemble will be confidently wrong. The edge and time splits test the ensemble's calibration directly, by checking whether true AME2020 masses fall inside its spread at the nominal rate.
Why not let width adapt during training instead of searching? Adaptive Width Neural Networks learns layer width jointly with the weights in a single run. It reports parameter parity with fixed-width models at up to 200 times fewer feed-forward parameters on translation, and over 50% pruning without accuracy loss. It is worth including as one entry in the benchmark. However, it optimizes a training likelihood, which is an interpolation objective, so it inherits the same selection blind spot. Its rank on the time split will show whether that matters.
Proposed Experiments and Decisive Checks
The first experiment builds the benchmark and reports the drip-line spread. For each element with measured neutron-rich isotopes, we plot the distribution of predicted two-neutron drip lines across the near-optimal random-split architectures. We compare it with seed-only spread and with the one-pair SkM*ext1 versus UNEDF0ext1 gap in magnesium. The claim is supported if, for a substantial fraction of chains, architecture spread exceeds one neutron pair while seed spread does not. It is refuted if architecture spread is consistently below one pair.
The second experiment is the ranking study. We compute Kendall's tau, overall and within the top decile, between each selection signal and time-split RMS on the 66 AME2020 nuclei. This set is small, so we will report bootstrap intervals and treat differences smaller than those intervals as null. The edge-split protocol is worth adopting only if its top-decile correlation with time-split error clearly exceeds that of random-split error. In addition, the architecture it selects must beat both LightGBM and the uncorrected WS4 baseline on time-split RMS.
The third experiment repeats the analysis across the four base models. This tests whether conclusions depend on how much physics the base model already captures. We expect architecture sensitivity to be largest on the crude liquid-drop base and smallest on WS4. If sensitivity persists even on WS4, the result bears on current practice rather than only on weak baselines.
The final check concerns the zero-cost scores. If any initialization-time score, including the physics-specific smoothness score, reaches top-decile correlation with time-split error comparable to edge-split validation, that would be the first evidence that such proxies carry information about extrapolation in scientific regression, the gap EZNAS and Abdelfattah et al. leave open. If none do, the negative result is itself useful. It would mean the NAS community's cheap scores measure trainability, not the inductive bias that governs behaviour at the edge of the nuclear chart.
References
- Studies of Deformed Halo Structures of 39Na and 42Mg
- Deep learning for nuclear masses in deformed relativistic Hartree-Bogoliubov theory in continuum
- Machine learning the nuclear mass
- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
- Zero-Cost Proxies for Lightweight NAS
- EZNAS: Evolving Zero-Cost Proxies For Neural Architecture Scoring
- On Random Weights and Unsupervised Feature Learning
- Rethinking Performance Estimation in Neural Architecture Search
- Validation of the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:mmultiscripts><mml:mrow><mml:mi>Be</mml:mi></mml:mrow><mml:mprescripts/><mml:none/><mml:mrow><mml:mn>10</mml:mn></mml:mrow></mml:mmultiscripts></mml:mrow></mml:math> Ground-State Molecular Structure Using <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline"><mml:mrow><mml:mmultiscripts><mml:mrow><mml:mi>Be</mml:mi></mml:mrow><mml:mprescripts/><mml:none/><mml:mrow><mml:mn>10</mml:mn></mml:mrow></mml:mmultiscripts><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mo>,</mml:mo><mml:mi>p</mml:mi><mml:mi>α</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mrow><mml:mmultiscripts><mml:mrow><mml:mi>He</mml:mi></mml:mrow><mml:mprescripts/><mml:none/><mml:mrow><mml:mn>6</mml:mn></mml:mrow></mml:mmultiscripts></mml:mrow></mml:mrow></mml:math> Triple Differential Reaction Cross-Section Measurements
- Adaptive Width Neural Networks