SignRot: LLM Quantization with Massive Outlier-Aware Sign-Adjusted Rotation

Jaehun Gim, Jaewoo Kim, Gyuwan Kim, Seo Yeon ParkarXiv 2026 · arXiv

TL;DR

SignRot is a lightweight, training-free enhancement for LLM quantization that addresses "bimodal fragmentation" caused by massive outliers during online rotations. By applying simple element-wise sign flips to the Hadamard transform, it collapses fragmented activation distributions into a quantization-friendly unimodal shape, significantly improving 4-bit model performance without adding computational overhead.

What Problem Does This Paper Address?

Quantizing Large Language Models (LLMs) to 4 bits is essential for efficient deployment, but "massive outliers"—activation channels with disproportionately high energy—often wreck quantization accuracy. To handle this, state-of-the-art methods like QuaRot use a Randomized Hadamard Transform (RHT) to "rotate" the activation space, spreading the outlier energy across all channels to make them easier to quantize.

However, the authors of SignRot discovered a critical flaw: when a single massive outlier dominates a token's activation, the RHT doesn't produce a smooth, bell-shaped distribution. Instead, it creates bimodal fragmentation—splitting the activations into two distant clusters (one positive, one negative) with a large empty gap in between.

Figure 1: Distributions of token activations in LLaMA2-7B layer 2 down_proj containing mas- sive outliers after rotation using randomized Hadamard and sign-adjusted Hadamard.

This fragmentation is disastrous for uniform quantization because the "empty space" between clusters inflates the quantization range, forcing a larger step size and wasting precious precision on values that don't exist. This is particularly problematic in the Feed-Forward Network (FFN) down-projection layers, where outliers are most severe and rotations must be applied "online" during inference.

Methods

The authors propose SignRot (Sign-adjusted Rotation), a method that identifies these problematic outliers and fixes the rotation matrix to prevent fragmentation.

  1. Identify: During a single calibration forward pass, the method identifies the specific channel index ($c_{max}$) containing the largest massive outlier.
  2. Adjust: Instead of using a purely random sign for the Hadamard transform, SignRot constructs a sign adjustment vector $\mathbf{n}$. This vector is designed to unify the signs of the row in the Hadamard matrix corresponding to the outlier channel.
  3. Collapse: By aligning these signs, the outlier's contribution becomes a uniform positive shift across all output channels, rather than a split between positive and negative. This collapses the bimodal distribution back into a unimodal, bell-shaped profile.
  4. Asymmetric Quantization: Because this alignment shifts the mean of the activations away from zero, SignRot pairs the rotation with asymmetric activation quantization. The quantization "zero-point" naturally absorbs this mean shift.
  5. Efficiency: The sign adjustment is mathematically "involutory," meaning its inverse can be fused into the model's weights offline. At inference time, SignRot adds only a simple element-wise sign flip, maintaining the speed of the Fast Walsh–Hadamard Transform (FWHT) without requiring expensive dense matrix multiplications.
Figure 3: Effect of sign adjustment on the random- ized Hadamard matrix under a massive outlier. Without adjustment (left), large outlier activation produces frag- mented outputs of mixed sign. With sign alignment via diag(n) (right), the outlier contribution becomes a uni- form positive shift, collapsing the bimodal distribution.

Datasets

The researchers evaluated SignRot across several model families, including LLaMA2 (7B, 13B, 70B), LLaMA3 (8B, 70B), Mistral-7B, and Qwen3-30B-A3B.

  • Language Modeling: Perplexity was measured on WikiText2 and C4.
  • Zero-Shot Reasoning: Accuracy was tested on six tasks: ARC (Easy & Challenge), HellaSwag, LAMBADA, PIQA, and WinoGrande.

Key Insights and Ideas

  • The Failure of the CLT: Standard rotation methods assume that the Central Limit Theorem (CLT) will result in a Gaussian distribution. The authors show that the "Lindeberg condition" for the CLT fails when one channel carries more energy than all others combined, which is exactly what happens with massive outliers.
  • Polarity Matters: Previous training-free methods were "sign-agnostic." SignRot proves that the specific polarity (sign) of the outlier is the key to fixing the distribution.
  • Training-Free Superiority: Surprisingly, this simple sign-flip logic allows a training-free method (QuaRot + SignRot) to outperform sophisticated, training-based methods like SpinQuant on large models like LLaMA3-70B.
  • Error Propagation: The paper highlights that quantization errors in massive outlier tokens (which act as "attention sinks") propagate to every other token in subsequent layers. Fixing just the first few layers where outliers appear provides a massive boost to the entire model.

Key Results

SignRot consistently improves performance across all tested benchmarks with negligible overhead:

  • LLaMA3-70B: Applying SignRot to QuaRot reduced WikiText2 perplexity from 6.16 to 5.88, surpassing the more complex SpinQuant (6.38).
  • Reasoning Accuracy: On LLaMA3-8B, SignRot improved average zero-shot accuracy by 2.83 points over the standard QuaRot baseline.
  • Inference Speed: In end-to-end tests on an RTX 3090, SignRot added only 0.43% latency to the total inference time while maintaining a 3.19× memory saving compared to FP16.
  • Quantization Time: A 70B model can be quantized in roughly 15 minutes on a single A100 GPU using GPTQ + SignRot.
Table 1: Comparison of the perplexity score on WikiText2 and C4 (PPL), and averaged accuracy on six zero-shot reasoning tasks (Avg. 0-shot) for LLaMA families. ‘+SignRot’ indicates that our method has been applied. Full results are in the Appendix F. Note that QuaRot uses symmetric activation quantization.

Limitations and Future Work

  • Single Outlier Focus: Currently, SignRot is optimized to handle the single most dominant outlier channel per layer. While it still helps when multiple outliers are present, it doesn't perfectly "fold" secondary outliers.
  • Channel Stability: The method relies on the fact that massive outlier channels are fixed across different inputs. In some later layers of specific models (like Mistral), the outlier channel can vary, which might reduce SignRot's effectiveness.
  • MoE Architectures: The efficiency of the underlying Hadamard transform decreases as the intermediate dimensions of a model get smaller, which is a common trait in Mixture-of-Experts (MoE) models.
  • Future Directions: The authors suggest extending SignRot to handle multiple outlier channels by assigning different Hadamard transforms to different channel groups.

Summary updated

Abstract

While quantization compresses LLMs and accelerates inference, outlier activations impede efficient low-bit representation. In particular, massive outliers cause substantial performance degradation. Although rotation-based methods that rely on randomized Hadamard transforms for online inference aim to mitigate this, they can produce fragmented distributions with empty bins when a massive outlier dominates a token's activation. This fragmentation severely undermines uniform quantization performance. Moreover, massive outliers concentrate in the down-projection input, where the preceding nonlinearity requires the rotation to be applied at inference time. This constraint prevents training-based approaches such as SpinQuant from learning the online rotation at this position, as a learned dense matrix cannot use the Fast Walsh-Hadamard Transform and would require a full matrix multiplication during inference. To address this, we propose SignRot (Sign-adjusted Rotation), which identifies massive outlier channels and constructs a sign adjustment vector that unifies the signs of the corresponding Hadamard row, collapsing the bimodal distribution into a unimodal profile amenable to quantization. Since the aligned outlier shifts the activation mean away from zero, we pair SignRot with asymmetric activation quantization, whose zero-point absorbs this shift. As a lightweight, drop-in improvement compatible with any Hadamard-based quantization method, SignRot requires only a single calibration forward pass and adds only element-wise sign flips at inference, without an additional matrix multiplication. Applying SignRot to QuaRot achieves a WikiText2 perplexity of 5.88 and an average zero-shot reasoning accuracy of 72.73, surpassing SpinQuant on the LLaMA3-70B model. Code is available at https://github.com/HYU-ARK-Lab/SignRot.
BibTeX
@misc{gim2026signrot,
  author = {Jaehun Gim and Jaewoo Kim and Gyuwan Kim and Seo Yeon Park},
  title = {{SignRot: LLM Quantization with Massive Outlier-Aware Sign-Adjusted Rotation}},
  year = {2026},
  eprint = {2611.16502},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16502},
}
Figure