Synthesizing Only the Wavelengths That Matter for Rare Classes

Hyperspectral cameras record hundreds of narrow wavelength bands for every pixel. In real scenes and lab spectrometry, a few material classes supply most of the labeled pixels and many classes supply only a handful. Two standard remedies both break down for those rare classes. Band selection, which keeps a few informative wavelengths and drops the rest, uses statistics pooled over the whole dataset, so it tends to keep the bands that separate common classes and discard the narrow absorption features that identify rare ones. Generative augmentation, which trains a diffusion model (a generator that learns to turn random noise into realistic samples by removing noise step by step) to produce extra rare-class samples, works but is expensive. Its cost grows with the full band count and the patch size, and an accuracy saturation result on imbalanced wafer maps suggests that much of that cost is wasted (Diffusion versus GAN Synthesis for Wafer Map Defect Classification).

The proposal treats each pixel's spectrum as a one-dimensional signal and splits it with a wavelet transform along the wavelength axis. The result is a smooth "continuum", the broad spectral shape driven largely by illumination, background, and scattering, plus fine-scale detail coefficients that carry narrow absorption features. We then pick a small set of detail coefficients separately for each rare class, chosen to separate that class from the class it is most often confused with. New rare-class samples are made cheaply: a real rare-class spectrum keeps its fine detail, its continuum is replaced by one taken from the abundant data, and only the few selected diagnostic coefficients are resampled from a tiny per-class model. If this works, rare-class pixels get realistic background variation from the abundant classes, and the only part that has to be generated is the low-dimensional part that is truly class-specific.

Core Claim

In long-tailed spectral data, scarcity hurts most in a small number of fine-scale spectral coefficients that are diagnostic for each rare class. The coarse spectral shape is largely shared across classes and plentiful. Our claim is that decomposing spectra this way, selecting diagnostic coefficients per rare class with a criterion that stays stable on few samples, and synthesizing only those coefficients while borrowing continua from abundant classes will match or beat full-spectrum diffusion augmentation on rare-class recall. The cost would be one to two orders of magnitude lower, because the generated dimension falls from hundreds of bands times a spatial patch to roughly ten to twenty numbers. This is a spectral-domain version of cut-and-paste context transfer, and the decomposition decides what gets cut and what gets pasted.

The claim makes three falsifiable predictions. Per-class selection should keep wavelengths that global selection drops. Continuum transplant should improve rare-class recall over plain oversampling. The resampling step should add value beyond transplant alone. We treat the third as genuinely uncertain (see below).

How This Relates to Prior Work

Four papers supply ingredients that we reuse. The per-class selection score adapts the Inter-Class Feature Differences criterion, which ranks wavelengths by the gap between class means divided by the within-class variances and picks 3 bands that reach 85.67% three-class accuracy on balanced maize-seed data, against 52.33% for the successive projections algorithm and 54.67% for random-forest selection (IFD). That work used exactly 300 seeds per class. Its variance estimate becomes unreliable when a class has 20 samples, and its edge disappears beyond about 15 bands. We change three things: the score is computed on wavelet detail coefficients rather than raw bands, separately for each rare class against its confuser, and with shrunken variances. The idea of mining the "confusing class" comes from DiSC-DS, which uses a pretrained classifier to find the class most similar to a rare food class and pushes diffusion samples away from it. We do the same mining but act on wavelengths instead of prompts, and we never run pixel diffusion. Continuum borrowing is the spectral analogue of Context-rich Minority Oversampling, which pastes rare foreground objects onto backgrounds from common-class images. It also parallels Oriented Calibration, which lets rare-class diffusion borrow structure from common classes during a middle window of denoising steps. In our method, the wavelet split replaces the timestep window as the place where common-class structure is allowed in.

Three other works motivate or constrain the design. The wafer study motivates it: higher-fidelity diffusion samples gave only 0.6–0.8 Macro-F1 points on vision transformers, and a GAN-plus-EfficientNet pipeline came within 0.5 points of the best result at one fifth of the compute. Fidelity bought at high cost is therefore a poor target. Latent-based Diffusion for Long-tailed Recognition cuts cost differently, by diffusing 64-dimensional encoder features. It treats all feature dimensions uniformly, and it depends on an encoder already trained on imbalanced data. We regard it as the main threat, not just a reference point. Balanced Data, Imbalanced Spectra shows that a class's error tracks the scale of its feature covariance eigenvalues, with an average correlation of −0.68. We use that offset as a diagnostic: good augmentation should shrink the inflated covariance spectra of rare classes rather than enlarge them. Finally, SCPSN groups correlated bands by global K-means. It represents the class-agnostic selection we argue against, and it is a natural point of comparison.

Research Program

Step 1: decomposition. Each spectrum $x$ is transformed with a discrete wavelet transform along wavelength into continuum coefficients $c$ (the coarsest approximation level) and detail coefficients $d$. Wavelet family and depth are tuned once on common classes only.

Step 2: per-class selection. For rare class $t$, we take its confuser $n(t)$ from the confusion matrix of a baseline classifier. Each detail coefficient $k$ is then scored as

$$ s_t(k) = \frac{|\mu_t(k) - \mu_{n(t)}(k)|}{\hat\sigma_t^2(k) + \sigma_{n(t)}^2(k)}, \qquad \hat\sigma_t^2(k) = \frac{n_t\,\sigma_t^2(k) + \tau\,\sigma_{\text{pool}}^2(k)}{n_t + \tau}, $$

The symbols are defined as follows. $\mu$ and $\sigma^2$ are the per-class mean and variance of coefficient $k$. $n_t$ is the number of training samples in class $t$. $\sigma^2_{\text{pool}}$ is the pooled within-class variance across all classes. $\tau$ is a shrinkage strength that pulls the unreliable rare-class variance toward the pooled one. Coefficients are added greedily by score, and a projection step in the style of the successive projections algorithm rejects candidates that are nearly collinear with ones already chosen. This gives a set $S_t$ of 5 to 20 coefficients.

Step 3: synthesis. A new sample of class $t$ is built in four moves. We take a real class-$t$ spectrum. We swap its continuum for one drawn from a pool of donor pixels, rescaled to match overall brightness. We redraw the coefficients in $S_t$ from a small per-class model; the default is a Gaussian with shrunken covariance, and the variant is a tiny diffusion model over $|S_t|$ dimensions. Finally, we invert the wavelet transform. For patch-based classifiers, one donor patch supplies continua for every pixel of the rare-class patch, so spatial structure in the continuum stays coherent.

There is an obvious risk: the continuum may itself carry class identity, as the broad vegetation red edge does. We measure this directly by training a classifier on continua alone. If continua turn out to be informative, donors are restricted to classes whose continuum-only predictions are confused with the rare class. If that restriction removes most of the benefit, the core claim is weakened, and we will report that.

Cost accounting. Every method is charged for generator training plus sampling, measured in GPU-seconds and in FLOPs, alongside the downstream classifier's training cost.

Baselines

Do nothing. The naive reference is a spectral-spatial classifier trained with plain cross-entropy on the long-tailed split using all bands. The classifier is the factorized (2+1)D network of EXNet, plus a 1D spectral-only network to isolate spectral effects. EXNet reports 99.61% overall accuracy on Indian Pines, 99.89% on Botswana, 99.60% on Salinas, and 97.35% on Pavia University, but only under its standard splits. We use spatially disjoint, artificially long-tailed training sets, where random pixel splits with overlapping patches cannot inflate scores, so those figures are a ceiling reference and not a target. We found no reported long-tailed hyperspectral classification result on Lacuna for EXNet or any other method.

Loss-based rebalancing. We run class-balanced reweighting, logit adjustment, and Seesaw Loss, which amplifies penalties when rare samples are misclassified as common ones and damps penalties against rare classes. Each method gets the same backbone and an equal 20-trial tuning budget.

Strongest competitors. The two strongest competitors are generative and are given the same classifier and tuning budget as our method. The first is full-patch diffusion augmentation, a class-conditional diffusion model on full-band patches, given the same number of synthetic samples per rare class. The second is LDMLR-style feature diffusion on the backbone's embeddings. Their only reported numbers are on natural images (86.29% top-1 on CIFAR-10-LT at imbalance factor 100, and 44.8% on ImageNet-LT), so we re-run them on our splits. Beating them means equal or higher rare-class macro-F1, within the seed-level confidence interval, at no more than one tenth of their generation cost. If LDMLR matches us at comparable cost, our claim reduces to an interpretable input-space method of similar quality, and we will say so.

Selection baselines. Selection is compared against global IFD, the successive projections algorithm, CARS, SCPSN-style band clustering, and a class-balanced global IFD that averages per-class scores with equal weights. The last of these isolates whether per-class, confuser-specific selection matters, or whether simply reweighting a global criterion is enough.

Ablations of our pipeline. We run four ablations: transplant alone with no resampling, resampling alone with no transplant, random coefficient sets of equal size, and raw bands instead of wavelet coefficients.

Why not just…?

Why not just rebalance the loss? Logit adjustment and Seesaw Loss cost nothing to generate, and on mild imbalance they may suffice. They cannot add spectral variability that the rare class never showed, however. With 20 training pixels, a rare class's decision region is fit to one illumination regime. The long-tailed latent-diffusion results show reweighting and logit adjustment trailing generative augmentation on CIFAR-LT, but that evidence comes from natural images. Our imbalance sweep from factor 10 to factor 100 decides the question for spectra. If loss rebalancing matches us at a factor of 10, we will report the factor at which synthesis starts to pay.

Why not just oversample with interpolation (SMOTE-style) in the full spectrum? Interpolating between two rare spectra stays inside the rare class's convex hull, so it never introduces new continua. It also mixes diagnostic and nuisance variation indiscriminately. Our resampling-only ablation is close to this alternative, restricted to the selected coefficients, and full-spectrum interpolation is run as its own baseline. If interpolation matches transplant, the context-transfer argument fails.

Why not just use the latent feature diffusion of LDMLR, which is already cheap? This is the real threat. A 64-dimensional feature diffusion model costs about as little as ours. Our bet is that it inherits an encoder trained on imbalanced data, which may already have folded rare classes into common ones. The covariance-offset diagnostic tests this directly: if LDMLR's synthetic features leave rare-class spectra inflated while ours shrink them, and recall follows, the distinction holds. If not, we concede the accuracy point and keep only the interpretability and backbone-independence point.

Why not just transplant continua and skip the generator? This may well work as well as the full method, and that would be a cleaner, simpler result rather than a failure. The transplant-only ablation is therefore a primary outcome, not a footnote. The generator is kept only if it adds measurable rare-class recall at fixed sample count.

Proposed Experiments

The main testbed is Indian Pines, Salinas, Pavia University, and Botswana. On each we build long-tailed training sets at imbalance factors 10, 50, and 100 by subsampling training pixels per class, with spatially disjoint test regions, and run five seeds. This responds to the single-seed limitation acknowledged in the wafer study. As a controlled small-sample test, we take the maize-seed Vis-NIR data from the IFD paper (900 seeds, three ages) and shrink one age class to 30 and then 10 samples. The decisive question there is whether shrunken per-class IFD recovers wavelengths near the reported 559.6, 631.2, and 640.5 nm that unshrunk IFD loses, and whether it stays above the reported 85.67% three-band accuracy as the class shrinks. The full-spectrum reference on the balanced data is 92.00%. Metrics are average per-class accuracy, macro-F1, recall on the rarest third of classes, overall accuracy, generation GPU-seconds, and the per-class covariance offset from Kaushik et al.

The headline figure plots rare-class macro-F1 against generation cost for every method. The proposal succeeds if our points dominate the full-patch diffusion curve by at least an order of magnitude in cost at matched recall, and if per-class selection beats class-balanced global selection at band budgets under 15. That budget range is where the IFD paper says ratio-based selection should help, and the 15-band-plus regime is where its advantage reportedly reverses, so we also report that regime. The proposal fails cleanly in three cases: if random coefficient sets of equal size do as well as selected ones, if continuum-only classifiers show the continuum is strongly class-informative so that transplanting it corrupts labels, or if loss rebalancing alone closes the gap at an imbalance factor of 100.