Abstract
In-context learning, i.e., learning from in-context samples, is an impressive ability of Transformer. However, the mechanism driving the in-context learning is not yet fully understood. In this study, we aim to investigate from an underexplored perspective of representation learning. The representation is more complex for in-context learning senario, where the representation can be impacted by both model weights and in-context samples. We refer the above two conceptually aspects of representation as in-weight component and in-context component, respectively. To study how the two components affect in-context learning capabilities, we construct a novel synthetic task, making it possible to device two probes, in-weights probe and in-context probe, to evaluate the two components, respectively. We demonstrate that the goodness of in-context component is highly related to the in-context learning performance, which indicates the entanglement between in-context learning and representation learning. Furthermore, we find that a good in-weights component can actually benefit the learning of the in-context component, indicating that in-weights learning should be the foundation of in-context learning. To further understand the the in-context learning mechanism and importance of the in-weights component, we proof by construction that a simple Transformer, which uses pattern matching and copy-past mechanism to perform in-context learning, can match the in-context learning performance with more complex, best tuned Transformer under the perfect in-weights component assumption. In short, those discoveries from representation learning perspective shed light on new approaches to improve the in-context capacity.
1. Introduction
In-context learning (ICL) is a defining capability of modern Large Language Models (LLMs), allowing them to adapt to new tasks using only a few examples provided in the prompt. Despite its success, the training process for ICL is often characterized by "learning plateaus"—extended periods where model performance remains stagnant before a sudden, sharp transition to high accuracy.
Understanding the mechanics behind these plateaus is crucial for improving training efficiency. Traditionally, the industry's answer to shortening these plateaus has been scaling up parameters and data. This study investigates these plateaus through the lens of representation learning, hypothesizing that ICL performance is driven by two distinct components: an "in-weights" component, which captures features derived solely from model parameters, and an "in-context" component, which manages the logic of using provided examples.
2. Methods and Synthetic Task Design
To disentangle these components, we developed a synthetic task based on the Shapes3D dataset. In this environment, objects possess independent factors such as color, shape, and scale. A Transformer model is tasked with performing ICL by identifying which factor is relevant based on context examples.
We define the Weights Component ($g_{weights}$) as the model's ability to extract basic features (e.g., "this is a blue cube") and the Context Component ($g_{context}$) as the mechanism that determines which feature to attend to based on the prompt. We utilize linear probes to monitor these internal representations throughout the training process.
To address the learning plateaus, we proposed three intervention strategies:
- Weights Warm-up: Pre-training the model on a simplified version of the task to stabilize the feature extractor.
- Mixed Training: Simultaneously training on tasks of varying complexity.
- Extra Loss: Incorporating an auxiliary supervised loss that encourages the model to predict underlying data factors directly.
3. Datasets
Our experiments utilized a diverse set of benchmarks:
- Shapes3D: A synthetic dataset allowing for precise control over task factors.
- SimpleFunction: A mathematical benchmark involving linear functions and Gaussian noise.
- SST-ICL: A natural language task based on the Stanford Sentiment Treebank.
- WordSelection: A custom NLP task requiring the model to select specific words based on contextual patterns.
4. Key Insights: The Representation Bottleneck
Our analysis reveals that learning plateaus are primarily a failure of feature extraction. During these periods, the model's "in-weights" component often stagnates or even deteriorates as it struggles to reconcile complex contextual logic with basic feature recognition.
We find that task complexity directly correlates with plateau duration. If the context is too difficult to decode, the Transformer lacks the "incentive" to refine its internal feature representations, as marginal improvements in feature extraction do not immediately translate to lower loss. This suggests that a robust in-weights component is a necessary foundation for effective in-context learning.
5. Results and Efficiency Gains
The proposed interventions successfully mitigated or eliminated learning plateaus across various tasks. In synthetic environments, Weights Warm-up led to immediate performance gains. In NLP tasks like WordSelection, the Extra Loss strategy significantly accelerated the "aha!" moment where the model masters the task.
Furthermore, we provide a theoretical proof by construction showing that a simple Transformer using pattern matching and a copy-paste mechanism can match the performance of more complex, best-tuned models, provided the in-weights component is perfectly learned. Empirically, these methods allow smaller models to achieve performance comparable to much larger baselines. For instance, a 6.69M parameter model trained with these strategies matched the accuracy of a traditionally trained 13.88M parameter model, while reaching peak performance in less than half the training time.
6. Conclusion and Future Directions
This work demonstrates that the "emergent" nature of ICL is deeply tied to the quality of internal representations. By focusing on the "in-weights" component, we can bypass traditional scaling hurdles and achieve high performance faster and with significantly smaller models. Future research will focus on developing unsupervised versions of these strategies that can identify and boost the "weights component" without needing labeled metadata.
Summary updated