Abstract
Code accompanying the arXiv submission "Characterizing out-of-distribution generalization of neural networks: application to the disordered Su-Schrieffer-Heeger model" including the generated datasets, and trained networks.
Introduction
Machine learning (ML) has become a powerhouse for detecting phases of matter in quantum physics. However, these models are often criticized as "black boxes" that may achieve 100% accuracy on clean, theoretical datasets but fail when applied to experimental data or models with disorder (randomness/noise). This phenomenon is a failure of Out-of-Distribution (OOD) generalization.
In the context of the Su-Schrieffer-Heeger (SSH) model—a prototypical topological system—the physically robust feature to identify is the presence of localized "edge states." The challenge is that neural networks (NNs) often find easier, non-physical "spurious correlations" in the training data that do not hold up when disorder is introduced. This paper investigates why this happens and how interpretability methods can build more reliable scientific ML tools.
Methods
The authors employ a shallow Convolutional Neural Network (CNN) to classify the topological phases of the SSH model. To look inside the black box, they use a multi-pronged interpretability suite:
- Class Activation Mapping (CAM): A technique that produces heatmaps showing which specific parts of the input data (which eigenstates) the CNN is "looking at" to make its decision.
- Dimensionality Reduction (PCA & UMAP): Analysis of the "latent representation"—the internal language the network uses in its final layers. By projecting high-dimensional activations into 2D space, researchers can see if the network perceives disordered data as similar to the training data.
- Thermometer Encoding: To combat "interpretation fragility" where noise makes CAM results nonsensical, the authors discretize the input data, mimicking a mercury thermometer, which stabilizes the gradients used for interpretability.
Datasets
The data consists of matrices where columns are the squared moduli of the eigenstates of the SSH Hamiltonian, sorted by energy. The Training Set uses the "clean" SSH model (zero disorder, $W=0$), while the OOD Test Set uses the disordered SSH model ($W > 0$), where Anderson localization makes eigenstates more complex.
Key Results and Insights
The "Edge State" Minority
An overwhelming majority (95%) of CNNs trained on clean data fail to generalize to disordered data. These networks are "lazy" learners, picking up "bulk" features that correlate with the phase in clean models but disappear under disorder. Only about 5% of randomly initialized networks "accidentally" learn to focus on the physically robust edge states. These are the only models that succeed OOD, achieving ~84% accuracy compared to ~55% for poorly-generalizing networks.
Latent Space as a Trust Metric
The authors found that if a network’s latent space (visualized via PCA) shows disordered data clustering in the same "neighborhoods" as the clean data, that network is much more likely to be reliable. If the disordered data forms a new, disconnected cluster, the network is essentially guessing.
Improving Generalization
By adding a tiny amount of "perturbative" disorder ($W=0.01$) to the training set, the success rate of producing a well-generalizing network jumped from 5% to 22%. CAM heatmaps clearly distinguish between "bad" networks (looking at the whole spectrum) and "good" networks (focusing specifically on the middle of the spectrum where edge states reside).
Discussion and Future Work
While Thermometer Encoding makes models more robust and interpretable, it comes with a high computational cost, increasing parameters from $10^3$ to $10^9$. Furthermore, CAM alone isn't a perfect predictor; some networks that focus on edge states still fail, suggesting that latent space analysis is a necessary second step for a complete "trust" profile. Future research should move toward physically motivated architectures or regularization techniques that force networks to ignore bulk correlations and focus on known topological invariants.
Summary updated