Symmetry-Protected Discretization in Differentiable Search over Crystal Hamiltonians

Many search problems relax a discrete choice into a continuous one so that gradients can be used, then round the result back to a discrete answer. The best-known case is differentiable architecture search, where a network blends every candidate operation and later keeps only the strongest one. That rounding step often destroys the property the search optimized. This is the discretization gap. A blended network at about 88% accuracy can fall to chance once snapped (Discretization-Aware Architecture Search). Condensed-matter physics has a precise vocabulary for properties that do and do not survive perturbation. Two energy bands that touch "by accident" separate under almost any small change. Two bands forced together by a nonsymmorphic symmetry stay together for every choice of parameters that respects the symmetry. A nonsymmorphic symmetry is a mirror or rotation combined with a shift by a fraction of the repeating lattice unit. The shift forces paired states across whole planes of momentum space (Node-surface and node-line fermions from nonsymmorphic lattice symmetries).

We propose to make this analogy literal and testable. We will build a differentiable search over tight-binding crystal models. These are small Hamiltonians whose discrete structure is the set of allowed electron hops between atomic sites. The search goal is a target band touching, such as a degenerate plane or line. Our claim is that the discretization gap here has an exact explanation. Unconstrained relaxation finds accidental, finely balanced touchings that rounding must destroy. Relaxation and rounding that both respect a nonsymmorphic group find protected touchings that rounding cannot destroy. A practical algorithm follows: tie the continuous choice weights across symmetry orbits, and round whole orbits at once. Physics also predicts which symmetry-breaking roundings should fail. This gives a testbed where the theory of the discretization gap can be checked against a ground truth, not inferred from accuracy curves.

Core Claim

The architecture-search literature names several causes of the gap: co-adaptation of parallel paths, sharp minima, and misleading selection weights. It does not have a theory of which properties survive rounding. Zhang et al. show that operations in a blended network compensate for each other's deficiencies (Interpreting Operation Selection in Differentiable Architecture Search). Zela et al. tie collapse to sharp curvature of the validation loss (Understanding and Robustifying Differentiable Architecture Search). Both describe the same thing a physicist would call an accidental degeneracy: a good objective value held up by fine-tuned cancellation between couplings. Removing a coupling during rounding is a finite perturbation, and finely balanced solutions do not survive finite perturbations.

In band theory, the condition for survival is known exactly. A band touching survives a perturbation if the touching is enforced by a symmetry that the perturbation preserves. Two materials show this cleanly. In strained SrIrO$_3$, a distortion that keeps a mirror plane but breaks an $n$-glide opens a 5 meV gap at a Dirac line (Strain-induced nonsymmorphic symmetry breaking...). In KHgSb, a distortion that breaks inversion but keeps the glide leaves the hourglass surface states intact (Robustness of topological states...).

Our claim has two parts.

  1. When the gap vanishes. In a differentiable search over hopping graphs, the gap for a band-touching objective is identically zero if two conditions both hold. The relaxed Hamiltonian must be equivariant under a space group whose nonsymmorphic generators force the target degeneracy. The rounding step must map symmetric models to symmetric models.
  2. Why the symmetry must be nonsymmorphic. A symmorphic (pure mirror or rotation) constraint is not enough. A mirror protects a touching only if the two bands carry opposite mirror labels, and that labeling depends on parameter values that rounding can flip. A glide or screw forces pairing from its algebra alone. Its square is a full lattice translation, so on the zone-boundary plane its eigenvalues are $\pm i$, and time reversal pairs them into a degeneracy regardless of parameters.

Research Program

Search space. We define a supernet over hopping terms. The supernet is the blended model containing every candidate. The building blocks are a fixed set of orbitals on sites of a chosen lattice. Each candidate hop $b$ (a pair of orbitals and a lattice vector up to a cutoff) gets a continuous presence weight $\alpha_b$ and a trainable amplitude $t_b$. The relaxed Bloch Hamiltonian is

$$H_\alpha(\mathbf{k}) = \sum_b \sigma(\alpha_b)\, t_b\, h_b(\mathbf{k}),$$

where $h_b(\mathbf{k})$ is the fixed matrix form of hop $b$ at crystal momentum $\mathbf{k}$, and $\sigma$ is a squashing function. The search objective rewards a target feature, such as the largest band gap over the $k_z=\pi/c$ plane being zero. It also rewards secondary targets such as bandwidth or a gap elsewhere in the zone. A cost term counts the number of hops present.

Rounding and the gap metric. Rounding keeps hops with large $\alpha_b$. The amplitudes are then retrained, exactly as DARTS retrains a snapped architecture (DARTS). We define the discretization gap as

$$\Delta_{\text{disc}} = \max_{\mathbf{k}\in P} \big(E_{n+1}(\mathbf{k}) - E_n(\mathbf{k})\big)\Big|_{\text{rounded, retrained}},$$

normalized by the total bandwidth. Here $P$ is the target plane or line, and $E_n$ and $E_{n+1}$ are the two bands meant to touch.

Why unconstrained search should fail. A degeneracy over a whole plane is not generic. Without symmetry it would require tuning infinitely many conditions at once. Unconstrained search can therefore only shrink the gap by balancing amplitudes. We expect the result to be a sharp minimum, in the sense of Zela et al., that retraining after rounding cannot restore.

Two ingredients of the proposal. The first is orbit tying. Hops related by an element of the chosen space group share one $\alpha$ and symmetry-related amplitudes, so the relaxed model is equivariant by construction. The second is orbit-wise rounding. Rounding keeps or removes entire orbits, so the rounded model stays in the symmetric subspace.

Space groups. The symmetry groups and target features come from the cited materials:

Group Character Source Expected behavior
$P6_3/mmc$ (No. 194) Screw axis and glides BaVS$_3$ / BaTaS$_3$ node-surface and node-line, Liang et al. Target touchings protected
$P6_3mc$ (No. 186) Inversion broken, glide kept KHgSb Should keep the glide-protected features
$Pbnm$ Orthorhombic, with $n$-glide SrIrO$_3$ Dirac line protected
$P112_1/m$ Monoclinic, mirror kept, $n$-glide lost SrIrO$_3$ Dirac line should gap
Symmorphic group with matching point group (No. 191) No glides or screws Control Separates the effect of having any symmetry from having nonsymmorphic symmetry

Numerical issue. Gradients of eigenvalues blow up exactly at degeneracies. We will therefore optimize smooth surrogates of the gap, such as a soft-minimum over sampled momenta of squared eigenvalue differences computed from characteristic-polynomial coefficients. We will report the exact gap only for evaluation.

Transfer back to architecture search. This arm is deliberately smaller and is a diagnostic, not a new search method. On NAS-Bench-201 (NAS-Bench-201) we define a per-edge fine-tuning index. It is the loss interaction from removing pairs of operations together versus one at a time, which is a second-order version of the perturbation selection in Rethinking Architecture Selection in Differentiable NAS. It plays the role of the coupling between two bands. We test whether this index predicts which edges lose the most accuracy when rounded. If it does, the accidental-versus-protected distinction is useful outside physics. If it does not, the physics result stands on its own as an exactly analyzable case of the gap.

Baselines

Naive reference. Plain DARTS-style relaxation with per-hop softmax or sigmoid weights, argmax-style rounding and retraining, and no symmetry information. This is the "do nothing" point. Its $\Delta_{\text{disc}}$ tells us how large the gap is in this domain.

Discretization-aware regularization. - Entropy-penalized relaxation following $DA^2S$, with penalties that push weights to 0 or 1 (Discretization-Aware Architecture Search). - Sigmoid weights with a zero-one loss following Fair DARTS. These are the strongest attempts to make the relaxed model "already discrete".

Exponentiated-gradient search. GAEA's mirror-descent updates on the simplex (Geometry-Aware Gradient Algorithms for Neural Architecture Search). We treat it as the strongest existing method because it produced the smallest post-rounding losses in the architecture-search literature. GAEA reports 94.10% CIFAR-10 and 73.43% CIFAR-100 test accuracy on NAS-Bench-201, against oracle values of 94.37% and 73.51%. These are the reference numbers for the NAS-Bench-201 arm. For tight-binding search no reported result found on Lacuna for any of these methods, so all physics-arm baselines are run by us.

Perturbation-based selection. PT selection applied to hops: mask each candidate, keep those whose removal hurts the objective most, and fine-tune between steps (Rethinking Architecture Selection in Differentiable NAS).

Regularized evolution. Regularized evolution directly over discrete hop sets, which NAS-Bench-Suite found to be among the most consistent searchers, with average rank 2.36 across 25 tasks (NAS-Bench-Suite).

Fairness rules. Every method gets the same candidate hop set, objective, amplitude-retraining budget, number of Hamiltonian evaluations, and hyperparameter search budget. Results are reported over at least 20 seeds, given the seed sensitivity noted for GAEA.

Ablations that isolate our ingredients. - Tied but individually rounded. Orbit-tied relaxation with per-hop rounding. This mimics strain that breaks a symmetry at rounding time. - Symmorphic only. Orbit tying under the symmorphic subgroup alone. - Symmetrize after rounding. Unconstrained search followed by projection onto the symmetric subspace.

What counts as winning. On nonsymmorphic targets, our method must keep $\Delta_{\text{disc}}$ at numerical zero (below $10^{-6}$ of bandwidth) in essentially all seeds while every unconstrained baseline leaves a finite gap. It must also match or beat the best baseline on the secondary objectives and on the number of hops.

Why not just…?

Why not just add an entropy penalty or use mirror descent so the relaxed model is near-binary? These methods shrink the change in the weights at rounding, but that is not the problem here. Even at nearly binary $\alpha$, an unconstrained near-touching over a plane depends on a precise balance of amplitudes. Retraining after rounding will not, in general, reach an exact degeneracy that symmetry does not require. The $DA^2S$, Fair DARTS, and GAEA baselines test this directly. If any of them reaches numerical-zero $\Delta_{\text{disc}}$ across seeds without symmetry, the core claim is wrong and we will say so.

Why not just impose whatever symmetry the material has, without caring whether it is nonsymmorphic? This is the most serious alternative. Equivariance alone might account for most of the benefit. The symmorphic-subgroup ablation settles it. Our prediction from mirror-eigenvalue protection is that symmorphic constraints sometimes protect the touching but lose it when rounding or retraining swaps the bands' mirror labels. Nonsymmorphic constraints should never lose it. If the two perform the same, the nonsymmorphic part of the claim fails and the contribution shrinks to "use equivariant relaxation".

Why not just symmetrize the model after rounding? Projection adds the missing members of each orbit, which changes the discrete structure after optimization. The added hops were never trained toward the secondary objectives. The symmetrize-after-rounding baseline measures the cost of this in objective value and hop count. If it matches our method, then orbit-wise rounding is unnecessary and we will report that.

Why not just enumerate or evolve discrete hop sets? For a four-band model with a small cutoff, orbit tying shrinks the space enough that enumeration may simply win. So we scale the orbital count and hop cutoff until enumeration becomes infeasible, and keep regularized evolution as a matched-budget baseline at every size. Black-box search is a real threat at small sizes. The proposal's value lies in the regime where it is too expensive, mirroring the compute argument for differentiable search versus NASNet's 1,800 GPU days reported in ASAP.

Proposed Experiments

Experiment 1: node-surface and node-line recovery. Start from four-band Wannier-style models of the BaVS$_3$/BaTaS$_3$ family (Node-surface and node-line fermions from nonsymmorphic lattice symmetries). Target the $k_z=\pi/c$ node-surface without spin-orbit coupling, and the fourfold node-lines along $A$–$L$ with it. Run all methods and record:

  • $\Delta_{\text{disc}}$;
  • the fraction of seeds where the degeneracy survives;
  • the Berry phase around the recovered line, which should be exactly $\pi$;
  • the number of hops.

Experiment 2: decisive symmetry-breaking check. Take the tied-relaxation model and apply roundings that each break one chosen symmetry.

  • A rounding that breaks the glide while keeping a mirror, as in the SrIrO$_3$ $Pbnm\to P112_1/m$ case, should open a gap at the Dirac line.
  • A rounding that breaks inversion but keeps the glide, as in KHgSb's $194\to186$ case, should leave the hourglass dispersion intact.

Agreement of a symmetry-based prediction with each rounding outcome, measured over many random symmetry-breaking roundings, is the main falsifiable result.

Experiment 3: robustness after rounding. Apply random symmetry-preserving perturbations of growing strength to the rounded models, and compare how fast the gap opens for each method. Protected solutions should show no gap opening. Accidental ones should open roughly in proportion to the perturbation.

Experiment 4: transfer to NAS-Bench-201. Compute the fine-tuning index on supernets trained with DARTS and GAEA, using naslib from NAS-Bench-Suite. Correlate it with the per-edge accuracy drop after rounding, and compare against curvature ($\lambda_{\max}$) as a predictor. Success means the index predicts the drop at least as well as curvature, at lower cost. Failure leaves the physics result as an exactly analyzable case of the discretization gap that does not carry over to image classifiers.