A recent result shows that large language models can be compressed to about one bit per weight with little loss by first running each weight row through a Haar wavelet transform, which splits the row into a smooth "average" part and a "difference" part, and then quantizing each part separately (HBLLM). The natural next step is to swap Haar for smoother, higher-order wavelets such as Daubechies. That swap rests on a hidden assumption. Wavelets only concentrate energy when neighboring entries of a signal are similar, and the order of input channels in a weight row is arbitrary. A network is unchanged if you shuffle its hidden channels consistently. Without a meaningful ordering, "low frequency" in a weight row has no meaning, and the smoothness properties that make Daubechies better than Haar in images and turbulence have nothing to act on.
This note proposes to make the ordering an explicit, optimized part of wavelet quantization. We first permute input channels so that adjacent channels are statistically similar, using calibration activations and weight-column similarity. Only then do we choose the wavelet basis and the number of decomposition levels for each layer. The claim is that the Haar-versus-Daubechies question and the depth question become meaningful, and win real accuracy below two bits, only after channel ordering is fixed. Under the native or a random order, all bases should behave like small block rotations and perform about the same.
Core Claim
A one-level Haar transform on pairs of coordinates is exactly a 2×2 Hadamard rotation applied blockwise. HBLLM's gain over spatial-domain binarization (BiLLM; perplexity 22.17 versus 6.71 on LLaMA-2-13B at about 1.08 bits) may therefore come mostly from local mixing plus band-wise grouping, which raises the number of distinct dequantized values (its CIQ metric, from 8 to 1024). It may owe little to frequency structure. If that is true, the vanishing moments of higher-order wavelets cannot help, because weight rows in their native channel order are not smooth polynomials along the index.
We predict two things. First, when channels are reordered by similarity, the approximation band of a multi-level wavelet captures much more than its share of row energy. Second, this compaction lets a small approximation band receive extra bits at negligible storage cost. The predicted payoff is a lower perplexity than HBLLM at a matched total bit rate, with all metadata counted, on brittle models such as LLaMA-3-8B, where HBLLM reports 20.09 and other one-bit methods exceed 50.
A second structural point removes the usual latency objection to longer filters. If a weight matrix is stored as $W = C\,T P$, the product with an activation is $Wx = C\,(T P x)$. Here $C$ is the quantized coefficient matrix, $T$ is an orthogonal wavelet transform, and $P$ is a channel permutation. The wavelet transform is therefore applied once to each activation vector and shared by every output row. It is never applied to each dequantized weight row. Its cost grows with hidden size times filter length per token, which is small next to the matrix multiply. This partly addresses the sequential-DWT latency problem flagged by Wavelet Convolutions for Large Receptive Fields.
Research Program
The work has three parts.
The first is a cheap diagnostic that needs no quantization. For every linear layer in LLaMA-2-7B, LLaMA-2-13B and LLaMA-3-8B, we measure the fraction of each row's energy in the level-$L$ approximation band. We do this for Haar, db2, db4, sym4 and an integer-lifting biorthogonal wavelet, and under four orderings: native, random, Hessian-diagonal order as used by GPTQ-style act-ordering, and similarity seriation. Seriation places correlated input channels next to each other by solving a traveling-salesman-style path problem on calibration activation covariance combined with weight-column cosine similarity.
Random ordering gives the null reference. Without structure, the approximation band at level $L$ should hold about $2^{-L}$ of the energy, because an orthogonal transform of an exchangeable signal spreads energy evenly. If seriation does not beat this null by a wide margin, the project's premise fails and we stop. The diagnostic also tests whether the native order already has structure in some layers. Query and key projections, whose rotary position-encoding dimensions come in paired, ordered frequencies, are a plausible case.
The second part is the quantizer. We reuse HBLLM's pipeline: saliency-based column selection, frequency-band grouping and intra-band mean sharing. We add the ordering $P$, a choice of basis, a depth $L \in \{1,2,3\}$, and allocation of extra bits to the shrinking approximation band. Permutations must be realizable exactly at inference. Every layer that reads the residual stream must share one permutation. That permutation commutes with the per-channel LayerNorm scales and can be folded into embedding and output projections. The inputs to the attention output projection and the MLP down-projection can be permuted independently, by folding the permutation into the rows of the value and up/gate projections. We will compare this exactly-folded variant with a looser variant that uses a separate permutation per layer plus a runtime gather. The comparison measures the price of the folding constraint.
The third part is per-layer selection. We choose basis and depth to minimize Hessian-weighted reconstruction error on calibration data under a global bit budget. We will also test whether the calibration-free heavy-tail spectral statistic used by AlphaQ predicts the same choices. A positive answer would remove a source of calibration bias.
Baselines
The naive reference is round-to-nearest binarization and plain spatial-domain binarization. Above these sits BiLLM. BiLLM combines Hessian-guided residual binarization of salient weights with optimal splitting of the remaining weights. It reports 8.41 WikiText-2 perplexity on LLaMA-2-70B at 1.08 bits, and HBLLM reports it at 22.17 on LLaMA-2-13B.
The strongest competitor is HBLLM itself: native ordering, one-level Haar, band grouping with mean sharing. Its reported numbers are WikiText-2 perplexity 6.71 on LLaMA-2-13B and 20.09 on LLaMA-3-8B at about 1.08 bits, with 73.8–88.8% of FP16 zero-shot accuracy retained across nine QA tasks. We will rerun HBLLM from its released procedure under our harness. All methods will share the same 128-sample WikiText-2 calibration set, the same single-GPU time budget, and a bit count that includes scales, means, permutation indices (amortized per layer) and any extra bits for the approximation band. Beating HBLLM means lower WikiText-2 and C4 perplexity and higher average zero-shot accuracy at equal or lower total bits, on at least LLaMA-3-8B and LLaMA-2-13B, with the gap larger than the run-to-run variance from three calibration seeds.
A rotation baseline replaces the wavelet with a global randomized Hadamard transform in the style of QuIP, or with the closed-form covariance-plus-Hadamard rotation of COAT. Each is combined with the same HBLLM grouping, so that only the transform differs. Two ablations isolate the key ingredient: the best basis and depth under random ordering, and Haar under seriated ordering. These separate the effect of ordering from the effect of basis.
Why not just…?
Why not just swap Haar for db4 in HBLLM and report the result? This is the obvious experiment, and we run it as an ablation. Our premise predicts a null or negative outcome. Longer filters spread a large salient weight across more coefficients, and without a smooth index they buy no compaction. The higher-order benefits documented in images and flows (WaveGAN, turbulence DWT analysis) rely on spatial continuity that weight rows lack by default. Without the ordering step, a negative db4 result could not be interpreted.
Why not just use a global Hadamard rotation, which WUSH proves optimal? WUSH proves Hadamard is the best data-agnostic mixing transform for equalizing dynamic range, and validates this at 4 bits. Our method is data-dependent through the ordering, so that optimality result does not rule it out. At about one bit, equalizing the range across all coordinates may matter less than concentrating energy into a small band that can afford more bits. This is a real threat. The Hadamard-plus-HBLLM-grouping baseline settles it directly, and if it matches seriated wavelets at equal bits, we will report that the wavelet adds nothing beyond grouping.
Why not keep native-order Haar and just add mixed-precision bit allocation across layers? Layer-level allocation, as in AlphaQ, and the heterogeneity results in Cherry on Top are complementary to our approach and could capture much of the gain. We will give HBLLM the same per-layer bit allocator. The claim survives only if ordering plus depth still wins within each layer's budget.
Proposed Experiments
We will first run the energy-compaction diagnostic across all three models. It is decisive and takes only hours. We then quantize LLaMA-2-7B, LLaMA-2-13B and LLaMA-3-8B at 1.08 and 1.5 total bits. Evaluation covers WikiText-2 and C4 perplexity and the zero-shot suite of PIQA, ARC-Easy, ARC-Challenge, HellaSwag, WinoGrande and BoolQ. We add GSM8K because low-bit damage concentrates in reasoning. The main grid crosses ordering, basis and depth.
We also measure end-to-end decoding latency. For this we use a Triton kernel that applies the activation-side permutation and wavelet before a standard low-bit matrix multiply. We compare its wall-clock time against HBLLM's purely theoretical 31.8%-of-FP16 estimate and against a Hadamard kernel.
The decisive check is a single comparison. Seriation must beat random and native ordering for db4 at depth 2 or more, and that combination must beat HBLLM on LLaMA-3-8B at matched bits. If the ordering effect appears in the compaction diagnostic but not in perplexity, then frequency structure is not what limits one-bit quality. That would itself be a useful finding about why HBLLM works.
References
- HBLLM: Wavelet-Enhanced High-Fidelity 1-Bit Quantization for LLMs
- BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
- Wavelet Convolutions for Large Receptive Fields
- AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
- QuIP: 2-Bit Quantization of Large Language Models With Guarantees
- COAT: COrrelation-Aware Orthogonal Transform for LLM Quantization
- WaveGAN: Frequency-Aware GAN for High-Fidelity Few-Shot Image Generation
- 103 ANALYSIS OF TURBULENT STRUCTURE USING DISCRETE WAVELET TRANSFORM
- WUSH: Near-Optimal Adaptive Transforms for LLM Quantization
- Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models