Catformer: Designing Stable Transformers via Sensitivity Analysis

Jared Quincy Davis, Albert Gu, Krzysztof Choromanski, Tri Dao, Christopher Ré, Chelsea Finn, Percy LiangICML 2021 · OpenReview

While Transformer architectures have revolutionized artificial intelligence, they remain notoriously "fussy" to train. Practitioners often rely on a delicate cocktail of custom learning rate schedules (like warmup), specific weight initializations, and careful placement of LayerNorm blocks to prevent the model from diverging.

The core issue is instability, but existing ways to measure it—such as tracking gradient norms or Lipschitz constants—are often too pessimistic or difficult to analyze. This instability is particularly punishing in Reinforcement Learning (RL), where data is noisy and non-stationary. In this paper, researchers introduce a theoretical lens to explain why certain architectures are more stable than others and use it to design a highly robust model: the Catformer.

Quantifying Instability: Sensitivity Analysis

To address the lack of formal metrics for training instability, the authors propose sensitivity analysis. They define "sensitivity" as the variance of the change in a model's output when its parameters are perturbed by a small, random amount.

By developing a set of "composition rules," researchers can now calculate the sensitivity of a complex architecture simply by looking at its individual building blocks. Crucially, the authors prove that for variance-preserving networks, gradient variance is proportional to the rate of change in sensitivity. This establishes sensitivity as a unified metric that bridges three previously separate notions of stability: activation variance, gradient variance (the vanishing/exploding gradient problem), and output change.

The Catformer Architecture

Based on their sensitivity analysis, the authors identify a "logarithmic divide": stable models typically have sensitivity that scales logarithmically with depth ($O(\log N)$), whereas unstable models scale linearly ($O(N)$). While Pre-Norm Transformers meet the stable criteria, the authors propose a new variant called the Catformer to push stability even further.

Instead of the standard residual connection ($x + F(x)$), the Catformer uses concatenation ($[x, F(x)]$).

Comparison of combination functions and normalization placements.

While both Pre-Norm and Catformer architectures exhibit $O(\log N)$ sensitivity, the Catformer has a unique property: its gradient variance scales proportionally to its weight variance. This alignment makes it theoretically more compatible with the dynamics of gradient descent. To keep parameter counts fair when comparing to standard Transformers, the Catformer uses projection matrices to manage the growing dimensionality caused by repeated concatenation.

Evaluation and Datasets

The paper evaluates the Catformer across three distinct regimes to test its theoretical benefits in practice:

  1. Synthetic Language Modeling: A "bit-reversal" memorization task designed to be difficult for shallow models, forcing the architecture to rely on depth.
  2. DeepMind Lab (DMLab-30): A suite of 30 challenging 3D visual navigation tasks based on the Quake III engine. These tasks involve partial observability, complex action spaces, and high-dimensional pixel observations.
  3. Robustness Tests: The authors test the models using "noisy" versions of these tasks, injecting gradient noise and switching from the Adam optimizer to standard SGD to test the limits of architectural stability.
DeepMind Lab (DMLab-30) 3D visual navigation tasks.

Key Results

The empirical results strongly validate the theoretical claims of the sensitivity analysis:

  • Superior RL Performance: On the challenging DMLab-30 benchmark, the Catformer outperformed the previous state-of-the-art architecture, GTrXL, by over 13%.
  • Extreme Robustness: The Catformer was remarkably stable in conditions where standard Transformers failed entirely. For example, it could be trained effectively using standard SGD (which usually fails for Transformers) and remained stable even when significant noise was injected into the gradients.
  • No Clipping Required: While GTrXL often required gradient clipping to prevent divergence in RL environments, the Catformer remained robust without any clipping interventions.

Limitations and Future Work

While the Catformer presents a significant leap in architectural stability, the authors note a few limitations. Because concatenation increases the size of the "state" passed between layers, the Catformer requires careful management of projection dimensions to keep the parameter count in check.

Additionally, while the results in RL and synthetic tasks are impressive, the paper does not evaluate the Catformer on massive-scale natural language pre-training (like GPT-style models). Testing whether these stability benefits translate to trillion-token regimes is a clear next step. Finally, the current sensitivity analysis treats internal modules (like Self-Attention) as somewhat of a "black box." Future work could refine the theory by looking deeper into the sensitivity of the attention mechanism itself.

Summary updated

Abstract

Transformer architectures are widely used, but training them is non-trivial, requiring custom learning rate schedules, scaling terms, residual connections, careful placement of submodules such as n...
BibTeX
@inproceedings{davis2021catformer,
  author = {Jared Quincy Davis and Albert Gu and Krzysztof Choromanski and Tri Dao and Christopher Ré and Chelsea Finn and Percy Liang},
  title = {{Catformer: Designing Stable Transformers via Sensitivity Analysis}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=56ql7upP2Fu},
}
Figure