Is It a Free Lunch for Removing Outliers during Pretraining?

Baohao Liao, Christof MonzarXiv 2024 · OpenReview

Abstract

With the growing size of large language models, the role of quantization becomes increasingly significant. However, outliers present in weights or activations notably influence the performance of quantized models. Recently, a novel softmax function was introduced aimed at pretraining models in an outlier-free manner, thereby enhancing their suitability for quantization. Interestingly, we observed that such an approach leads to performance degradation in full precision. Building on this insight, we enhance the method by ensuring its normalization is invariant to sequence length, a crucial factor for bridging the gap between pretraining and fine-tuning. Moreover, this improved method also facilitates successful pretraining of causal language models.

Introduction

Large Language Models (LLMs) are notoriously difficult to deploy on consumer hardware due to their size. Quantization—converting high-precision 16-bit weights and activations into 8-bit or 4-bit integers—is the standard solution. However, "outliers" (extreme values in the data) act as a wrench in the works. Because quantization scales values into a fixed range, a single massive outlier forces the remaining values to be squeezed into a tiny, low-resolution space, destroying the model's accuracy.

Recent research suggested a "pre-emptive" strike: using a Clipped Softmax (CS) function during pretraining to prevent these outliers from ever forming. While this makes the models "quantization-friendly," it often causes significant performance degradation in full-precision (FP16) models. This paper introduces Normalized Clipped Softmax (NCS), a novel technique that stabilizes the training process and ensures that outlier-free models remain high-performing across different sequence lengths and model architectures.

Methods: The Evolution of Normalized Clipped Softmax

The authors propose Normalized Clipped Softmax (NCS) to fix the flaws in the original Clipped Softmax.

  1. The Flaw in CS: In the original Clipped Softmax, the sum of the probabilities is no longer guaranteed to be 1. More importantly, this sum changes depending on the sequence length ($T$). This creates a "mismatch" when a model trained on short sequences is asked to perform on long ones (or vice versa), leading to representation distortion.
  2. The NCS Solution: The authors introduce a normalization constant ($\beta$) that makes the sum of the clipped probabilities invariant to the sequence length.
  3. Causal Support: Unlike the original CS, which was designed for bidirectional models like BERT, the authors tailored NCS to work with causal (generative) LLMs like OPT by accounting for the triangular attention mask where sequence length effectively varies for every token.

Key Insights: Attention and Sequence Length

Outliers often emerge from "no-op" attention heads—heads that focus almost all their probability on a few common tokens (like periods or commas) to effectively "skip" an update. Clipping the softmax forces the model to distribute its attention more healthily, but if not normalized correctly, it breaks the model's internal logic.

Visualization of Attention Probability, Value, and Product matrices showing how attention is distributed across tokens.

The most surprising finding was that the original Clipped Softmax's performance was tied to the specific sequence length used during pretraining. This explains why previous models looked good in lab settings but failed when fine-tuned on real-world tasks with varying lengths. NCS is significantly less sensitive to hyperparameter tuning than the original CS, reducing the need for expensive grid searches.

Key Results

The models were pretrained and evaluated using a combination of BookCorpus and English Wikipedia (Wiki-40b), with performance measured on the GLUE Benchmark and Wikipedia validation sets.

  • Recovering FP16 Performance: On the GLUE benchmark, BERT-base with the original Clipped Softmax scored an average of 68.2. By using the proposed NCS, the score jumped to 73.8, significantly closing the gap with the vanilla model (81.7) while maintaining excellent quantization properties.
  • Quantization Success: Both CS and NCS achieved near-perfect 8-bit quantization (W8A8). For BERT-base, the perplexity dropped from a broken 4612.6 (vanilla) to a functional 4.95 (NCS).
  • Causal LLM Breakthrough: The original CS actually increased perplexity for the OPT-125M model. NCS, however, successfully trained the model with a perplexity (15.86) nearly identical to the vanilla version (15.84).
MLM Accuracy vs Sequence Length for BERT-small and BERT-base, comparing Vanilla, CS, and NCS methods.

Limitations and Future Work

While NCS is a massive improvement over CS, there is still a performance gap between the outlier-free models and vanilla models in full precision. The study primarily focused on models under 350M parameters; the behavior of massive "frontier" models (7B+ parameters) might differ. Future work will involve evaluating NCS on vision and speech transformers to prove its universality across domains.

Summary updated

Abstract

With the growing size of large language models, the role of quantization becomes increasingly significant. However, outliers present in weights or activations notably influence the performance of quantized models. Recently, \citet{qtransformer} introduced a novel softmax function aimed at pretraining models in an outlier-free manner, thereby enhancing their suitability for quantization. Interestingly, we observed that such an approach leads to performance degradation in full precision. Building on this insight, we enhance the method by ensuring its normalization is invariant to sequence length, a crucial factor for bridging the gap between pretraining and fine-tuning. Moreover, this improved method also facilitates successful pretraining of causal language models.
BibTeX
@misc{liao2024is,
  author = {Baohao Liao and Christof Monz},
  title = {{Is It a Free Lunch for Removing Outliers during Pretraining?}},
  year = {2024},
  eprint = {2402.12102},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.12102},
}
Figure