Machine-learned corrections to nuclear mass models are judged almost entirely by one number: the root-mean-square deviation (RMSD) between predicted and measured binding energies, averaged over a couple of thousand nuclei. That number hides what matters most. The nuclei that drive astrophysical predictions are neutron-rich, often strongly deformed, and mostly unmeasured. A correction can lower the average error by fitting well-measured nuclei near stability while getting no better, or getting worse, on deformed or neutron-rich ones. Computer vision ran into the same problem and developed a fix. It tags each test example with "factors of variation", such as occlusion or pose, and reports error separately for each factor. We propose to bring that practice to relativistic nuclear structure. We also adopt its strictest form, in which each test slice varies one physical attribute while the other attributes are held fixed, so that a model's error can be attributed to deformation, to neutron excess, or to plain distance from measured data, and not to some mixture of the three.
The core claim is specific and testable. On nuclear mass tables, the physical attributes are strongly correlated with each other and with extrapolation distance. Deformed nuclei sit mid-shell. Neutron-rich nuclei sit far from where masses have been measured. Because of this, naive binning of residuals assigns errors to the wrong cause, and aggregate RMSD ranks correction methods differently from how they perform on newly measured exotic nuclei. A matched-slice error profile should fix both problems. It is cheap to compute and needs no new annotation, because every attribute is either computed by the physics model itself or is a simple function of proton and neutron number.
What prior work does and leaves open
Two papers supply the models we will dissect, and both report only aggregate scores.
Deep learning for nuclear masses in deformed relativistic Hartree-Bogoliubov theory in continuum uses DRHBc, a fully relativistic mean-field theory that handles axial deformation, pairing and coupling to unbound states. They train a neural network on DRHBc results for even-proton nuclei plus measured masses from the 2020 Atomic Mass Evaluation (AME2020). The aim is to fill in the expensive odd-proton entries. The network lowers the RMSD against experiment from 1.433 MeV for raw DRHBc to 0.842 MeV, and reaches 0.725 MeV on newly measured $^{88,89}$As. The same paper finds that spherical and deformed relativistic masses differ by more than 5 MeV for some isotopes, and that this difference shifts predicted r-process abundances by up to two orders of magnitude for mass numbers 80–120. A local discrepancy of that size is exactly what a global RMSD cannot show.
Machine learning the nuclear mass trains a gradient-boosted tree model (LightGBM) on the residuals of four non-relativistic models, using AME2016. Its best result reduces the WS4 model's RMSD from about 0.302 MeV to about 0.170 MeV. It explains the model with SHAP feature attributions, which estimate each input's average contribution to a prediction. The authors note that the method degrades on light nuclei, but they do not quantify where else it degrades or why.
We use both papers as the systems under test. They are not competitors to our protocol.
The method comes from vision. ImageNet-X defines an error ratio, the error on a factor divided by the overall error. With it, the authors show two things that aggregate accuracy hides. Augmentations that help one factor hurt others, a "spill-over" effect. Larger models gain accuracy overall while their error ratios on the hardest factors grow. TracKlinic adds a purity rule: each test sub-sequence contains only one challenge factor. It shows that entangled factors compound failures non-linearly and that some factors that look important are negligible once isolated; illumination, for example, causes under 1% of failures. Our contribution is the nuclear counterpart of the purity rule. Video clips can be cut so that only one factor is present. Nuclei cannot be cut, so we build purity statistically by matching nuclei on all attributes except the one under study.
Research Program
Every nucleus in the evaluation set receives the following attribute vector:
- the quadrupole deformation $\beta_2$ reported by DRHBc;
- the relativistic deformation gap $\Delta_{\rm def}$, defined as the DRHBc mass minus the spherical RCHB mass;
- the neutron excess $(N-Z)/A$, where $N$ is neutron number, $Z$ proton number and $A=N+Z$;
- the distance to the nearest proton and neutron magic numbers;
- the odd-even class;
- the extrapolation distance, meaning the number of isotopic steps to the nearest nucleus in the training set.
The relativistic deformation gap is the attribute that ties the study to relativistic modeling. It is a model-internal measure of how much the deformed relativistic treatment matters for that nucleus.
For a target attribute $a$, we build a matched slice. We stratify nuclei by $a$ and reweight each stratum so that its joint distribution over the remaining attributes matches a common reference. This uses coarsened matching on binned attributes, with bootstrap confidence intervals throughout. The matched error ratio is
$$R_a(k) = \frac{\mathrm{RMSD}_w(\text{stratum } k \text{ of } a)}{\mathrm{RMSD}(\text{all})},$$
where $\mathrm{RMSD}_w$ is the weighted RMSD under the matching weights. $R_a(k)>1$ means the model is worse than average in stratum $k$ when every other attribute is held fixed. The naive error ratio is the same quantity computed without weights. The gap between the naive and matched ratios measures how much confounding the naive version contains.
The analysis asks three questions, each borrowed from vision.
The first is attribution. Does the DNN correction from Choi et al. reduce error on strongly deformed nuclei once extrapolation distance is held fixed? Or does its apparent gain on deformed nuclei come entirely from those nuclei being close to the training data?
The second is spill-over. Do residual corrections, the LightGBM corrections of the four base models and the DNN correction of DRHBc, lower matched error ratios on some attributes while raising them on others?
The third is capacity. As we grow the correction network or the boosted-tree ensemble, does the aggregate RMSD fall while the matched ratio on the hardest stratum, which we expect to be high extrapolation distance at large neutron excess, rises? That would mirror the EfficientNet finding in ImageNet-X.
A practical product follows from these answers: a ranked map of which odd-proton nuclei most need a direct DRHBc calculation, because the correction network is least trustworthy there.
Baselines
The naive reference is the current standard: a single global RMSD on a random hold-out of AME nuclei, computed for each model. It is the number that every paper in this area reports. For the relativistic system, the reported values are 1.433 MeV for raw DRHBc and 0.842 MeV for the DNN-corrected table (Choi et al.).
The strongest existing correction method is LightGBM residual learning on WS4, which reports an RMSD of about 0.170 MeV on AME2016 (Machine learning the nuclear mass). We will reproduce it with the paper's ten features. We will also apply the same LightGBM residual learner to DRHBc, which the original paper did not do. All corrected models will be trained on identical training splits with an identical hyperparameter search budget. That way, differences in their slice profiles reflect the base model and the correction method, not the amount of tuning.
The strongest existing diagnostic is global SHAP attribution on these same models. We will compare its implied ranking of attributes against our matched ranking.
The ablation that isolates our key ingredient is naive marginal binning: the unweighted ratio, equivalent to reading residual maps on the nuclear chart. If matched and naive profiles agree in ranking and sign for every attribute, the purity step adds nothing, and we will say so.
The protocol beats these baselines if its profiles, computed only from pre-split data, predict model behavior on genuinely new nuclei better than they do. The two tests are defined under Decisive Experiments below.
Why not just…?
Why not just use SHAP, as the LightGBM paper already does? SHAP reports how much each input feature moves the model's own output on average. It does not report where the model's error against experiment is large. Its contributions are also blurred across correlated features, which is precisely the situation here, with deformation, shell distance and neutron excess moving together. The LightGBM paper's own light-nucleus failure was found by inspecting errors, not attributions. We test SHAP directly by checking whether its feature ranking predicts the matched error ratios. If it does, the cheaper tool suffices for ranking, and our contribution narrows to effect sizes and confidence intervals.
Why not just plot residuals on the nuclear chart, as nuclear physicists routinely do? This is the most serious alternative, and we treat it as the main ablation, not a straw man. Chart maps are naive slices. They show where errors occur, but they cannot separate a deformation effect from an extrapolation effect when the two coincide geographically. The experiment settles the question. If matched and naive attributions agree, the purity rule is unnecessary in this domain, much as illumination turned out to be a non-issue in TracKlinic. We would report that as a negative result.
Why not just report RMSD on a time-split hold-out of newly measured nuclei? Choi et al.'s 0.725 MeV on the arsenic isotopes and LightGBM's check on 66 newly measured AME2020 nuclei already move in this direction. A time split is honest about extrapolation. However, it rests on very few nuclei and gives one number, which says nothing about which attribute causes the error. We use the time split as the arbiter of whether our profiles are predictive, not as a rival.
Decisive Experiments
The data are AME2016 and AME2020, plus the DRHBc and RCHB tables used by Choi et al. The comparison set is DRHBc, DRHBc with the DNN correction, DRHBc with a LightGBM correction, and LightGBM-corrected LDM, DZ, WS4 and FRDM, all retrained by us under a matched budget.
The first decisive check is temporal. We fit every model and compute every slice profile on AME2016 only. We then evaluate on the nuclei newly measured in AME2020. The test is whether matched error ratios on the extrapolation and neutron-excess strata, together with the global RMSD, rank the models' errors on the new nuclei better than the global RMSD alone. We will report rank correlation with bootstrap intervals, and we will state plainly if the new-nucleus set is too small to tell the two apart.
The second check is a synthetic extrapolation split, which supplies more test nuclei. We hold out the outermost measured nuclei along each isotopic chain and repeat the comparison.
The third check targets the relativistic question directly. It asks whether the matched error ratio rises with the deformation gap $\Delta_{\rm def}$ for the corrected DRHBc table, with extrapolation distance held fixed. If it does, the network is failing exactly where deformation physics matters for the r-process window that Choi et al. identified.
Two limitations are acknowledged in advance. First, $\beta_2$ and $\Delta_{\rm def}$ are derived from DRHBc itself, so we will cross-check them against the deformation implied by the non-relativistic models. Second, light nuclei are too sparse to match reliably, so they will be reported separately rather than forced into strata.