Breaking through the learning plateaus of in-context learning in Transformer

Jingwen Fu, Tao Yang, Yuwang Wang, Yan Lu, Nanning Zheng

TL;DR

This paper identifies that "learning plateaus"—long periods of stalled progress during Transformer training for in-context learning—are primarily caused by a dysfunction in the model's ability to represent individual inputs (the "weights component"). By introducing targeted training strategies like weights warm-up and auxiliary losses, the authors demonstrate how to bypass these plateaus, enabling more computationally efficient and "eco-friendly" development of powerful AI models.

What Problem Does This Paper Address?

In-context learning (ICL) is the remarkable ability of Large Language Models (LLMs) to learn new tasks simply by seeing a few examples in their input prompt, without any parameter updates. However, training models to possess this skill is notoriously expensive.

A major culprit is the learning plateau: a phase during training where the model's performance remains flat for thousands of iterations before suddenly jumping in a "transition process." Traditionally, researchers have shortened these plateaus by scaling up model size and compute, which is environmentally and financially costly. This paper seeks to understand the internal mechanics behind these plateaus and find ways to "break through" them without simply throwing more hardware at the problem.

Figure 1. A: Examples of the in-context learning tasks. Examples of (1) comes from Alayrac et al. (2022), Examples of (2),(3),(4) come from Brown et al. (2020). B: Illustration of learning plateaus and transition pattern. We evaluate the in-context learning ability of Pythia 13B model (Biderman et al., 2023) trained on pile dataset (Gao et al., 2020) using WordSelection task (Detail in Appendix B) during the training process.

Methods

The authors propose a conceptual framework that decomposes the model's internal representations into two parts:

  1. Weights Component: The part of the representation solely dependent on the model's learned parameters (e.g., how it "sees" or extracts features from a single image or word).
  2. Context Component: The part influenced by the relationship between the prompt and the provided examples.

To study these, they developed a synthetic task based on the Shapes3D dataset. In this task, a Transformer must predict a property of a "prompt" image (like its color or shape) based on context examples that define which property is currently relevant.

Figure 2. Synthetic task. In the task, Transformer is required to predict the label of xp given context examples sc. The images from the 3D Shapes dataset are synthesized based on six factors. The output factor is determined by the context. In this case, we provide two sequences of factors: ”object color” and ”object shape,” respectively.

Using linear probes to monitor these components, they discovered that during plateaus, the "weights component" often deteriorates or stagnates. Based on this, they proposed three breakthrough strategies:

  • Weights Warm-up: Initially training the model on a simpler task with a fixed mapping to refine its feature extraction before introducing complex, context-dependent tasks.
  • Mixed Training: Training on a 50/50 mix of simple (fixed mapping) and complex (random mapping) tasks simultaneously.
  • Extra Loss: Adding an auxiliary supervision signal that forces the model to explicitly predict the underlying factors of the input, directly boosting the weights component.

Datasets

  • Shapes3D: A synthetic vision dataset used to create controlled ICL experiments with six independent factors (color, shape, scale, etc.).
  • SimpleFunction: A synthetic task involving linear functions to validate findings in a non-visual domain.
  • SST-ICL: A version of the Stanford Sentiment Treebank adapted for in-context sentiment analysis.
  • WordSelection: A custom NLP task requiring the model to select a specific word from a list based on context patterns.
  • The Pile: Used to evaluate the pre-trained Pythia 13B model to demonstrate that plateaus are a real-world phenomenon.

Key Insights and Ideas

  • Plateaus are a "Weights Dysfunction": The most striking finding is that in complex tasks, the model's ability to represent individual inputs actually gets worse during the plateau. The model lacks the "incentive" to refine its vision/feature extraction because it hasn't yet figured out the logic of how to use the context.
  • The Bottleneck is Representation, not Logic: The authors proved mathematically and experimentally that if a model has a "perfect" weights component (it can see the features clearly), it only needs a few extra layers to perform ICL perfectly. The difficulty of ICL training lies in learning to extract the right features while simultaneously learning how to relate them.
  • Linear Correlation: There is a near-linear relationship between the quality of the weights component and the duration of the plateau. Better feature extraction leads to a faster "aha!" moment for the model.

Key Results

The proposed methods significantly improved training efficiency:

  • Efficiency Gains: A smaller 6.69M parameter model trained with the mixed method achieved accuracy comparable to a 13.88M parameter model trained normally, but in less than half the time (20 epochs vs. 50 epochs).
  • Plateau Elimination: In synthetic tasks, a brief weights warm-up of just a few epochs was enough to completely eliminate the initial learning plateau.
  • NLP Success: On the SST-ICL task, the mixed training approach significantly outperformed standard training after approximately 35 epochs, proving the method generalizes beyond synthetic data.
Comparison of the three proposed methods showing significant mitigation of learning plateaus and faster accuracy gains.

Limitations and Future Work

  • Scope of Plateaus: This research focuses specifically on plateaus in in-context learning. It does not explain plateaus seen in standard supervised learning, such as the "grokking" phenomenon.
  • Conceptual vs. Physical: The decomposition into weights and context components is conceptual for analysis; in a real Transformer, these processes are physically intertwined across layers.
  • Scaling Up: While tested on models up to 34M parameters and validated against 13B parameter checkpoints, future work is needed to see if these auxiliary losses can be integrated into massive-scale LLM pre-training to reduce global energy consumption.

Summary updated

Abstract

In-context learning, i.e., learning from context examples, is an impressive ability of Transformer. Training Transformers to possess this in-context learning skill is computationally intensive due to the occurrence of learning plateaus, which are periods within the training process where there is minimal or no enhancement in the model's in-context learning capability. To study the mechanism behind the learning plateaus, we conceptually seperate a component within the model's internal representation that is exclusively affected by the model's weights. We call this the "weights component", and the remainder is identified as the "context component". By conducting meticulous and controlled experiments on synthetic tasks, we note that the persistence of learning plateaus correlates with compromised functionality of the weights component. Recognizing the impaired performance of the weights component as a fundamental behavior drives learning plateaus, we have developed three strategies to expedite the learning of Transformers. The effectiveness of these strategies is further confirmed in natural language processing tasks. In conclusion, our research demonstrates the feasibility of cultivating a powerful in-context learning ability within AI systems in an eco-friendly manner.
BibTeX
@inproceedings{fu2024breaking,
  author = {Jingwen Fu and Tao Yang and Yuwang Wang and Yan Lu and Nanning Zheng},
  title = {{Breaking through the learning plateaus of in-context learning in Transformer}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=2K87GFLYWz},
}
Figure