A growing line of work replaces the last, learned layer of a classifier with a fixed matrix whose class vectors form a simplex equiangular tight frame (ETF): unit-length vectors, one per class, with every pair at the same angle. The motivation is neural collapse, the observation that well-trained networks drive class means toward this arrangement anyway. These fixed heads are credited with better long-tailed accuracy, less client drift in federated learning, and faster convergence. But a simplex ETF bundles three properties that nobody has pulled apart. Its vectors have equal length, they are as spread out as possible (low "coherence," the largest absolute cosine between two class vectors), and the frame is tight, meaning it covers every feature direction with equal energy. The simplex ETF also only exists when the feature dimension is at least the number of classes minus one. That condition fails exactly where long-tailed recognition is hardest: iNaturalist 2018 has 8,142 classes against a 2,048-dimensional ResNet-50 feature.
We propose a closed-form family of fixed classifiers built from rows of a discrete Fourier transform (DFT) matrix. The family contains the simplex ETF as one member. It keeps every class vector at exactly equal length by construction, and it exposes two independent knobs: which frequencies are used, which controls coherence, and how each frequency row is scaled, which controls how evenly the frame spreads energy across feature directions. It works for any number of classes and any dimension. Using this family inside one controlled, seed-audited long-tailed pipeline, we ask which of the three bundled properties produces the reported gains. We also ask what the right fixed head is when no simplex ETF exists.
Core Claim
The property that constant-modulus matrices provide, where every entry has the same magnitude, turns "equal class-vector length" into a structural invariant rather than something enforced by normalization. That frees the other two properties to be varied cleanly. Pick $d$ distinct frequencies $r_1,\dots,r_d$ from $\{0,\dots,C-1\}$ and positive scales $s_1,\dots,s_d$. The classifier vector for class $k$ is then
$$ w_k = \big(s_1\,\omega^{r_1 k},\;\dots,\;s_d\,\omega^{r_d k}\big), \qquad \omega = e^{2\pi i/C}, $$
where $C$ is the number of classes and $d$ is the feature dimension. Every $w_k$ has squared length $\sum_j s_j^2$, which does not depend on $k$. The frame operator, the $d\times d$ matrix that measures how much total energy the class vectors put along each feature direction, is $C\,\mathrm{diag}(s_j^2)$, so the scales set its spectrum directly. The cosine between classes $k$ and $l$ depends only on their difference $m = k-l$:
$$ |\langle w_k, w_l\rangle| = \Big|\sum_{j=1}^{d} s_j^2\,\omega^{r_j m}\Big|, $$
that is, the Fourier transform of the scale profile placed on the chosen frequencies.
Three special cases matter. Taking all nonzero frequencies ($d=C-1$) with equal scales gives the simplex ETF. Choosing the frequencies as a difference set, where such a set exists, gives an ETF with $C\gg d$ that meets the Welch bound. Other frequency sets give harmonic tight frames whose coherence can be minimized by a cheap combinatorial search over frequency sets.
This is the row-scaling construction of DFT Spectral Tetris from Spectral tetris fusion frame constructions, which built unit-norm frames with any prescribed spectrum. There it was a tool for existence proofs in signal processing. We use it as an experimental instrument. The complex-to-real problem that paper leaves open has a simple answer in our setting. Pairing frequencies $r$ and $C-r$ yields a cosine row and a sine row. If both rows share one scale, each class vector's squared length still gets $s^2(\cos^2+\sin^2)=s^2$, so equal lengths survive in real arithmetic at the cost of using dimensions in pairs.
Our claim is empirical and falsifiable. We expect low coherence to be the property doing the work, with tightness secondary and equal length alone insufficient. We further expect a coherence-optimized harmonic head to beat the ad hoc fixed heads currently used when $C > d+1$. If the experiments show otherwise, for example that a random equal-length head matches everything, that is the more important finding, because it would mean the ETF story in this literature is mostly about freezing the head.
Relation to Prior Work
Several papers motivate the question by using fixed or guided ETF heads without isolating why they help. No Fear of Classifier Biases freezes a synthetic simplex ETF in federated learning and reports 56.08% versus FedAvg's 44.48% on CIFAR-10 at Dirichlet α=0.05. Early Exit with Disentangled Representation and Equiangular Tight Frame fixes ETF heads at intermediate BERT layers. Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem derives equal per-class loss from the simplex ETF target and gains 6.26% to 7.08% over cross-entropy on CIFAR-100-LT. None of the three varies the target geometry, so none can say which property matters. We vary exactly that.
Guiding Neural Collapse takes a different route. It keeps the simplex ETF shape but rotates it toward the features by solving a Riemannian problem every batch. Its backward pass is costly enough that it is dropped at scale. That work adapts orientation; ours changes shape at zero training-time cost.
Exploring deep neural networks via layer-peeled model constrains the design. Minority collapse, where tail-class classifier vectors merge, is a failure of angles, not lengths. So equal-length heads alone should not cure it. This is why a random equal-length head is our key ablation.
Neural Collapse in Multi-Task Learning shows that task heads settle into mutually orthogonal ETFs. Harmonic frames give a natural analogue: disjoint frequency sets are orthogonal blocks.
On the frame-theory side, Prime tight frames characterizes when a harmonic frame splits into smaller tight sub-frames. We use that result for one secondary experiment. The evaluation methodology comes from the standardized-evaluation literature, which repeatedly finds that claimed gains shrink under controlled pipelines:
- BARS: tuned decade-old recommender baselines win.
- TFB: the "drop last" evaluation bug in time-series forecasting. - A Sober Look at the Unsupervised Learning of Disentangled Representations: random seeds explain more variance than the choice of method.
This last finding is why we report variance decompositions and not just means.
Research Program
We will build a single long-tailed training harness on top of the RIDE codebase. It already supports cross-entropy, LDAM, decoupled classifier retraining (cRT), and τ-normalization on CIFAR-100-LT, ImageNet-LT, and iNaturalist. The harness will fix the data splits, augmentation, schedule, and backbone, and it will evaluate on the full test set with no dropped final batch. Every configuration will be run with five seeds under an equal tuning budget.
Inside the harness, the head is swappable. The harmonic family is generated in closed form. The frequency set is chosen by a greedy-plus-local-swap search that minimizes the coherence formula above. That search costs $O(C \cdot d)$ per candidate and runs once, offline.
The analysis has two parts. The first fixes the frequency set and sweeps a one-parameter scale profile from flat (tight) to strongly uneven. This changes only the spectrum, with lengths held exactly equal. The second fixes flat scales and sweeps frequency sets of increasing coherence, including deliberately bad ones such as consecutive frequencies. This changes only coherence, with lengths and tightness held fixed. Crossing the two sweeps gives a small grid in which each cell differs from its neighbour in exactly one geometric property. That cleanliness is what the construction buys over ad hoc heads.
For each cell we record four groups of measurements:
- accuracy overall and on the many-, medium-, and few-shot splits;
- expected calibration error;
- the collapse of tail-class feature means, measured by their average pairwise cosine;
- the fraction of across-run variance attributable to seed versus head geometry.
A secondary experiment on CIFAR-100 uses its 20 superclasses. Following the decomposition result in Prime tight frames, we build a harmonic head that is not prime and whose tight sub-frames align with superclasses, and compare it to one whose sub-frames cut across them. This tests whether the head's block structure can encode class hierarchy for free.
Baselines
The naive reference is a standard learnable linear head trained with cross-entropy in the same harness. Improving Calibration for Long-Tailed Recognition reports this configuration at 41.2% top-1 with 27.5% ECE on CIFAR-100-LT at imbalance factor 100. We re-run it rather than copy the number.
The second baseline is a learnable cosine classifier, whose class vectors are renormalized to unit length at each step. It isolates "equal length" from "fixed." It is run with the same schedule and a temperature tuned on the same budget as our head's single scale hyperparameter.
The third is the fixed simplex ETF, as used in FedETF-style work. Where $C > d+1$ it cannot exist, and we use the most common substitute: a fixed random frame whose vectors are normalized to unit length. This random equal-length frame is also our key ablation. It matches our head in being fixed and equal-length, and differs only in coherence and tightness. Beating it is the minimum condition for the construction to matter.
The strongest existing competitors come in two groups. The first is geometric: Implicit ETF from Guiding Neural Collapse, which reports 72.15% on balanced CIFAR-100 with ResNet-50 at epoch 50 versus 58.47% for standard training. We run it where its per-batch solver is affordable, with the same backbone and schedule. The second is the long-tailed state of the art on CIFAR-100-LT at imbalance factor 100:
- GLMC from Global and Local Mixture Consistency Cumulative Learning, reported at 55.88% top-1;
- RIDE from Long-tailed Recognition by Routing Diverse Distribution-Aware Experts, reported at 49.1%;
- MiSLAS, reported at 47.0% top-1 and 4.83% ECE.
These methods change the loss, the augmentation, or the architecture, not just the head. So the fair comparison is to add our head to each of them, alongside comparing against them. Beating a baseline means a gain larger than the seed-level 95% confidence interval under an identical pipeline, not a gain against a published number from a different pipeline.
Why not just…?
Why not just normalize a learned classifier (cosine classifier)? Equal length is trivial to impose by renormalization. If the harmonic head's only effect were equal length, the cosine baseline would match it, and we would say so. The layer-peeled analysis in Exploring deep neural networks via layer-peeled model predicts that a learned head, normalized or not, can still let tail vectors drift together in angle. The tail-class cosine measurement tests this directly. This alternative is a real threat on balanced data, and we expect the gap, if any, to appear only on few-shot classes.
Why not just freeze a random unit-norm matrix? In high dimension, random vectors are already fairly incoherent, so this is the most dangerous alternative, especially at iNaturalist scale. The coherence sweep settles it. If accuracy is flat across coherence levels between random and optimized harmonic frames, structure does not matter, and the paper reports that the ETF benefit reduces to freezing the head. The prediction for the head to matter is a monotone drop in few-shot accuracy as coherence rises.
Why not just use the simplex ETF and add better reweighting or experts? Where the simplex ETF exists, it is a point in our family, so our head cannot be worse than it by construction unless a different point is chosen. The real question is whether moving off that point helps, which the grid answers. The combination experiments with GLMC and RIDE test whether head geometry adds to loss- and data-side methods or is absorbed by them. If it is absorbed, the practical value is limited to regimes with $C > d+1$.
Decisive Checks
Three outcomes decide the proposal.
First, on iNaturalist 2018 with ResNet-50, where no simplex ETF exists in 2,048 dimensions, a coherence-optimized harmonic head must beat the random equal-length fixed head and the learned cosine head on few-shot accuracy by more than seed noise. RIDE's reported 72.6% on iNaturalist is the external reference; there, we report our head added on top of RIDE.
Second, on CIFAR-100-LT and ImageNet-LT, the crossed sweep must show that coherence explains more of the variance in few-shot accuracy than the spectrum does. Otherwise the tightness knob, not the coherence knob, is the finding.
Third, the variance decomposition must show that head geometry explains more variance than seed choice. If seed dominates, as it did for disentanglement methods, the honest conclusion is that fixed-head geometry is a second-order effect. That negative result is itself worth publishing under a standardized protocol.
References
- Spectral tetris fusion frame constructions
- No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier
- Early Exit with Disentangled Representation and Equiangular Tight Frame
- Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of View
- Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame
- Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training
- Neural Collapse in Multi-Task Learning
- Prime tight frames
- BARS: Towards Open Benchmarking for Recommender Systems
- TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods
- A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation
- Improving Calibration for Long-Tailed Recognition
- Global and Local Mixture Consistency Cumulative Learning for Long-tailed Visual Recognitions
- Long-tailed Recognition by Routing Diverse Distribution-Aware Experts