Adaptive energy reference for machine-learning models of the electronic density of states

Wei Bin How, Sanggyu Chong, Federico Grasselli, Kevin K. Huguenin-Dumittan, Michele CeriottiPhysical 2024

Predicting the electronic density of states (DOS) is a holy grail for computational materials science. It tells us how electrons are distributed across energy levels, which in turn dictates a material's conductivity, optical properties, and more. While Machine Learning (ML) has accelerated these predictions, it often stumbles over a subtle physical reality: in bulk materials, there is no "absolute" zero for energy.

A new paper by Wei Bin How and colleagues introduces a "self-aligning" framework that treats the energy reference as a learnable parameter, significantly boosting the accuracy and transferability of DOS models.

TL;DR

The authors propose an adaptive energy reference scheme that optimizes the energy alignment of each material structure during the training process. By allowing the model to "self-align" spectral features, they achieve up to a 50% reduction in error and significantly better transferability to larger system sizes compared to standard fixed-reference models.

What Problem Does This Paper Address?

When we calculate the DOS using Density Functional Theory (DFT) for an infinite bulk system, the absolute energy scale is ill-defined because there is no "vacuum level" to serve as a universal zero. Traditionally, researchers pick a reference point, like the average Hartree potential ($V_H$) or the Fermi level ($E_F$).

The problem? These choices are somewhat arbitrary and can vary wildly across a dataset. If two similar atomic environments have their DOS shifted by 2 eV because of a global reference choice, an ML model—which usually looks at local atomic neighborhoods—gets confused. It sees the same local structure but is told to predict different energy spectra. This "artificial noise" makes models harder to train and less accurate.

Methods

The core innovation is a self-aligning loss function.

In a standard setup, the model tries to minimize the Mean Squared Error (MSE) between the predicted DOS and the DFT-calculated DOS at fixed energy levels. In the proposed Adaptive scheme, the authors introduce an optimizable parameter, $\Delta_A$, for each structure $A$ in the dataset.

  • The Loss Function: Instead of forcing the model to match a fixed target, the loss function allows the target DOS to shift along the energy axis during training to find the best alignment with the model's current prediction.
  • Architecture: They tested this using both linear models and Multilayer Perceptrons (MLPs).
  • Descriptors: They used the Smooth Overlap of Atomic Positions (SOAP) power spectrum to represent the local chemical environments.
  • Efficiency: To keep the shifting process computationally cheap, they represented the DOS using cubic Hermite splines, allowing for fast, continuous shifts and derivative calculations.

Datasets

The authors validated their approach across four diverse datasets:

  1. Silicon (Si): 927 structures including diamond, liquid, amorphous phases, and surfaces.
  2. Hydrogen (H): Over 7,000 structures representing high-pressure solid and liquid phases.
  3. Barium Titanate ($BaTiO_3$): 840 structures covering four different ferroelectric phases.
  4. Gallium Arsenide ($Ga_xAs_{1-x}$): A binary system with varying stoichiometry and temperatures.
  5. Transferability Test: A specific set of Silicon vacancy defects (training on 63-atom cells and testing on 215-atom cells).

Key Insights and Ideas

  • Local vs. Global: Local descriptors (like SOAP) are great at identifying chemical motifs but blind to global shifts caused by the choice of energy reference. The adaptive scheme bridges this gap by removing the "burden" of predicting global shifts from the local model.
  • Feature Matching: The "aha moment" comes from looking at the results: the adaptive scheme automatically aligns prominent spectral features (like the deep 3d band in Gallium). When these "anchor points" are aligned, the model can focus its capacity on learning the finer, more physically relevant details of the DOS.
  • Multimodal Datasets: For materials like $BaTiO_3$, the $\Delta_A$ values showed distinct clusters (bimodal distribution). This suggests the adaptive scheme is effectively identifying different physical phases within the dataset and aligning them appropriately.

Key Results

  • Consistent Improvement: The adaptive model outperformed both $V_H$ and $E_F$ references across all datasets.
  • Massive Error Reduction: In the $Ga_xAs_{1-x}$ dataset, the adaptive scheme reduced the Root Mean Squared Error (RMSE) by roughly 50% compared to the Fermi-level baseline. For $BaTiO_3$, the improvement was about 25%.
  • Superior Transferability: When training on small Silicon cells (63 atoms) and testing on large cells (215 atoms) with vacancies, the adaptive model reduced error by 38% to 59% compared to fixed-reference models. This suggests the self-aligning approach learns more "universal" physics that isn't tied to the specific size of the simulation box.
  • Derived Quantities: The model also performed better at predicting secondary properties like the distribution of excitations ($X(\theta)$), even though these are technically independent of the energy reference.

Limitations and Future Work

  • Training Time: The authors note that while the joint optimization of $\Delta_A$ and model weights is efficient, it is still slightly more complex than standard training.
  • Dataset Dependency: The degree of improvement depends on how "misaligned" the original dataset is. For simple systems like pure Silicon, the gains are smaller (~10%) compared to complex alloys or insulators.
  • Beyond DOS: The authors suggest this concept of "optimizing an ill-defined reference" could be applied to other properties, such as predicting the spatially resolved density of states or the Hamiltonian of bulk structures.

Paper Citation: Wei Bin How, et al. "Adaptive energy reference for machine-learning models of the electronic density of states." arXiv:2407.01068v4 [cond-mat.mtrl-sci], 2025.

Summary updated

Abstract

Machine learning methods for predicting electronic density of states often assume that the model predictions and targets share the same absolute energy reference. However, this overlooks a subtle point, that the absolute energy reference cannot be defined for infinite bulk systems due to the conditionally convergent nature of electrostatic potentials. This paper introduces a training framework that uses a self-aligning loss function to provide an adaptive energy reference during training. Models trained using this framework outperform those relying on fixed conventional internal energy references, like the Fermi level or average Hartree potential. The paper also sheds insights on how the adaptive reference enhances model performance and identifies the conditions under which they are the most effective.
BibTeX
@inproceedings{how2024adaptive,
  author = {Wei Bin How and Sanggyu Chong and Federico Grasselli and Kevin K. Huguenin-Dumittan and Michele Ceriotti},
  title = {{Adaptive energy reference for machine-learning models of the electronic density of states}},
  booktitle = {Physical},
  year = {2024},
}
Figure