Low-power sensors increasingly skip rebuilding the full signal and classify directly from a handful of compressed linear measurements. Every such system still has to turn those measurements into bits, and on the cheapest hardware that means one or two bits per measurement. The usual engineering shortcut is to treat quantization error as small, independent white noise. That assumption is provably false once measurements are redundant or bits are few. In that regime, adding more measurements stops helping, because the errors become deterministically correlated and the error settles at a floor. Sigma-delta modulation is the standard fix. It is a converter that feeds each quantization error back into the next sample, which pushes the error into a part of the signal that can be filtered away. Every existing sigma-delta design, though, pushes the error away from the signal, because its goal is reconstruction.
We propose to push the error away from the decision instead. A classifier reads only a few directions of the measurement vector: the ones along which classes differ. The feedback filter can therefore hide quantization error in the much larger space of directions the classifier ignores. That space is far bigger than the out-of-band region that reconstruction-oriented designs rely on, so the claim is that classification error should fall with oversampling faster, and start falling sooner, than under reconstruction-oriented noise shaping at the same bit budget. The gain should grow as the discriminative subspace gets smaller relative to the signal dimension.
Core Claim
Consider an oversampled measurement vector $y=\Phi x \in \mathbb{R}^N$ that passes through a $b$-bit quantizer inside a feedback loop. The output can be written $q = y + G e$. Here $e$ is the per-sample quantizer error, bounded by $\|e\|_\infty \le \Delta/2$ with step size $\Delta$, and $G$ is the lower-triangular noise-transfer operator set by the feedback filter, with unit diagonal. A downstream classifier $f$ is sensitive to perturbations of $q$ roughly through its input Jacobian. We summarize that sensitivity as $W = \mathbb{E}_x[J(x)^\top J(x)]$, where $J(x)$ is the Jacobian of the class-margin with respect to the measurements. We choose the filter by solving
$$ \min_{G} \; \max_{\|e\|_\infty \le \Delta/2} \; e^\top G^\top W G\, e \quad \text{subject to a no-overload constraint on } G, $$
and compare against the reconstruction design, which uses the same objective with $W$ replaced by the dual-frame reconstruction weighting. The worst-case form is chosen on purpose. It gives a guarantee that does not rely on the white-noise hypothesis, which is exactly the assumption that fails in this regime. We also compare it to the average-case (trace) version, which does assume white noise, to measure how much that assumption misleads filter design.
Our prediction is as follows. With $W$ having effective rank $k \ll d$, where $d$ is the signal dimension, the task-shaped loop reaches a given classification error at markedly lower oversampling than reconstruction-shaped sigma-delta. Both beat dithered memoryless quantization. Undithered memoryless quantization hits a classification floor.
Where This Sits
Motivating work. White Noise Hypothesis for Uniform Quantization Errors proves that quantization errors across redundant frame channels cannot be independent. The white-noise model holds only for linearly independent vectors as the step size shrinks. The Performance of PCM Quantization Under Tight Frame Representations shows that memoryless (PCM) quantization of unit-norm tight frames hits a reconstruction floor of order $\delta^{(d+1)/2}$ no matter how large the redundancy. Neither paper asks what happens to a classifier reading those measurements, and our first experiment does.
Work that supplies the method. Performance Analysis and Optimal Filter Design for Sigma-Delta Modulation via Duality With DPCM shows that the optimal sigma-delta feedback filter is the minimum-mean-square-error predictor of a "virtual" process, and it handles overload with an excess-rate argument. We reuse that machinery but swap the reconstruction error weighting for the classifier-sensitivity weighting $W$. In the time-series setting this becomes a spectral weighting: the noise is steered out of discriminative frequency bands rather than out of the signal band.
Work that constrains the design. Generalization Error Analysis of Quantized Compressive Learning proves that classification under quantized Gaussian projections depends on the debiased variance of inner products near the decision boundary, not on raw distortion. It reports that a 1-bit Lloyd-Max quantizer beats a 3-bit uniform one in high-similarity regions. This supports our choice of a task-dependent objective. Their analysis covers only memoryless quantizers, unit-sphere data and dimensionality reduction ($k<d$), never redundancy or feedback. Compressive classification and Bounds on the Number of Measurements for Reliable Compressive Classification show that classification needs fewer measurements than recovery, because only inter-class separation matters. This is the geometric reason a small discriminative subspace should exist. Both assume Gaussian channel noise rather than deterministic quantization error.
Direct compressive classifiers. Simple Classification using Binary Data classifies directly from sign measurements with layered lookup tables. It needs mean-centered data, and it leaves dithering untested. C²SP-Net and TransCL learn binary sensing matrices and classify without reconstruction. However, both keep the measurements themselves in floating point. Our work puts the quantizer, and its feedback, back into this pipeline.
Research Program
The work has three stages.
Stage 1: establish the classification floor. On synthetic Gaussian-mixture classes observed through unit-norm tight frames, as in the Wang–Xu and Reboredo settings, we sweep redundancy $N$ and bit depth. We measure MAP classification error under undithered memoryless quantization, dithered memoryless quantization, and the two sigma-delta designs. Theory is tractable here. For a linear readout $L$ of rank $k$, the error seen by the classifier is $LGe$. We will derive how the achievable worst-case norm $\|LG\|_{\infty\to 2}$ scales with $N$ for $r$-th order shaping. We will state it against the reconstruction case, where the known rates come from bounding the error through the reconstruction dual frame.
Stage 2: build the design procedure for learned classifiers. We alternate three steps: train the classifier on quantized measurements, estimate $W$ from its Jacobians on held-out data, and re-solve for the filter under the overload constraint. The overload constraint is essential, because aggressive shaping destabilizes 1-bit loops. Following Ordentlich–Erez, we bound the $\ell_1$ norm of the filter taps and add a small excess rate where needed. We also fold the filter taps into training via a straight-through gradient, and compare this with the alternating solver.
Stage 3: move to real signals where oversampling is natural. EEG is sampled in time, and sigma-delta converters are already standard in biosignal front ends. We apply the method to CHB-MIT seizure prediction. The input is a C²SP-Net-style learned compression matrix whose outputs are now quantized to 1–3 bits. For images, we use MNIST and Extended YaleB with random projections at 1–2 bits. These have direct comparison points in Needell et al.
Baselines
Naive reference. Undithered memoryless (PCM) uniform quantization of the same measurements, followed by the same classifier retrained on the quantized inputs. This is what a system designed under the white-noise assumption would deploy. It is where we expect a classification floor with increasing $N$, and beating it is necessary but not sufficient.
Strongest memoryless baseline. Dithered memoryless quantization with a Lloyd-Max quantizer. Dithering restores near-independence of errors and should give roughly $1/N$ decay. Lloyd-Max is the quantizer that Li & Li show is best for classification among memoryless options. For 1-bit data we also run the Needell et al. layered lookup table with dithered thresholds. Their reported figure on MNIST is over 90% accuracy with 200 1-bit measurements and 18 layers, which we will reproduce before comparing. Li & Li give no numeric accuracies: no reported result found on Lacuna for arcene, BASEHOCK or orlraws10P accuracy.
Strongest feedback baseline. Reconstruction-optimal sigma-delta, meaning the Ordentlich–Erez filter of the same order, with the same bits, the same overload constraint and the same downstream classifier. This baseline isolates the key ingredient. The only difference is whether the filter minimizes reconstruction error or classifier-weighted error. A second ablation shapes noise into a random subspace of the same rank as $W$. That tests whether the gain comes from discriminative directions specifically, rather than from any low-rank weighting.
Upper reference. Full-precision measurements. On CHB-MIT, C²SP-Net reports 92.5% accuracy uncompressed and 91.9% at 1/16 compression with floating-point measurements. That 91.9% is the ceiling for our quantized variants at the same compression.
Fairness and success criterion. All methods share data splits, classifier architecture, total output bits per signal and hyperparameter search budget. Beating the baselines means the following. At matched bits, the task-shaped loop must reach the dithered-PCM and reconstruction-sigma-delta accuracy with at least a factor-of-two lower oversampling, or give a strictly steeper log-log slope of error versus $N$. On CHB-MIT at 2 bits, it must close most of the gap to 91.9%.
Why not just…?
Why not just dither memoryless quantization? Dither breaks the deterministic correlations that cause the Wang–Xu floor, and One-Bit Sigma-Delta MIMO Precoding uses it for exactly that reason. It is a serious competitor. Dither only buys independent errors, however, so classifier-visible error falls at best like $1/N$, whereas shaped feedback should fall polynomially faster. If dithered Lloyd-Max matches task-shaped sigma-delta at the redundancies that matter in practice (say $N/d \le 8$), the proposal loses its practical case. The Stage 1 and Stage 3 sweeps report exactly this crossover.
Why not just learn the sensing matrix and quantizer thresholds end to end with a straight-through gradient? This extends C²SP-Net and TransCL, and it can shift quantizer thresholds toward decision boundaries. It is still memoryless, though. No choice of thresholds lets one sample's error cancel another's, which is the mechanism behind noise shaping. The Wang–Xu floor applies to any memoryless per-channel rule on a redundant frame. We run this learned-memoryless baseline with the same training budget. If it matches our method, that would mean the learned matrix is itself avoiding redundancy, and we would report that as a negative result.
Why not just use standard sigma-delta and let the classifier adapt? This is the reconstruction-sigma-delta ablation. Reconstruction shaping spends its noise budget clearing the whole signal band, including directions the classifier never reads. The prediction is that the gap between the two designs shrinks to zero as the rank of $W$ approaches $d$. Seeing that trend would confirm the mechanism. Seeing no gap at small rank would refute it.
Why not just reconstruct first, then classify? Reconstruction adds cost that direct classification avoids. C²SP-Net reports cutting multiply-accumulates from 1050K to 71.88K. Under undithered PCM, reconstruction is also bounded below by the Wang–Xu floor. We include a reconstruct-then-classify pipeline built on sigma-delta measurements for completeness, but we do not expect it to change the conclusion.
Decisive Checks
The central figure plots classification error versus oversampling ratio on log-log axes for all five quantization schemes. It uses matched bits, on synthetic mixtures with discriminant rank $k \in \{1, 4, 16\}$ at $d=64$. The claim is supported if three things hold: undithered PCM plateaus, dithered PCM decays with slope near $-1$, and task-shaped sigma-delta outpaces reconstruction-shaped sigma-delta by a margin that widens as $k/d$ shrinks. A second check compares worst-case and white-noise-based filter designs at 1 bit. If they differ, the white-noise assumption is actively misguiding design. On CHB-MIT, the decisive number is accuracy at 1/16 compression with 1-bit and 2-bit measurements against the 91.9% floating-point reference, with overload rate reported for every loop. The main risk is loop instability under aggressive shaping at 1 bit. If the overload constraint erases the gain, that constraint is the finding.
References
- White Noise Hypothesis for Uniform Quantization Errors
- The Performance of PCM Quantization Under Tight Frame Representations
- Performance Analysis and Optimal Filter Design for Sigma-Delta Modulation via Duality With DPCM
- Generalization Error Analysis of Quantized Compressive Learning.
- Compressive classification
- Bounds on the Number of Measurements for Reliable Compressive Classification
- Simple Classification Using Binary Data
- C2SP-Net: Joint Compression and Classification Network for Epilepsy Seizure Prediction
- TransCL: Transformer Makes Strong and Flexible Compressive Learning
- One-Bit Sigma-Delta MIMO Precoding