Pretrained language models possess a remarkable capability known as In-Context Learning (ICL): they can adapt to entirely new tasks simply by observing a few examples in their input prompt, without undergoing any parameter updates. For years, this behavior was treated as a black box. However, a growing body of research has begun to uncover the internal mechanics of this phenomenon, revealing that a frozen model's forward pass acts as a dynamic runtime environment that constructs and executes structured internal algorithms.
In-Context Learning as Implicit Optimization
To understand why in-context learning works, we can look at it through the lens of optimization theory. When a model processes a prompt containing input-label pairs $(x_1, y_1), (x_2, y_2), \dots$, how does it use this data to predict the next token?
The answer is that the model's internal layers mathematically mimic traditional machine learning optimization algorithms. In a foundational theoretical proof, Arvind V. Mahankali et al. demonstrated in One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention that a single layer of linear self-attention trained on linear regression tasks is optimized to perform exactly one step of pre-conditioned gradient descent. The attention mechanism uses the key and value matrices to store the sample covariance of the prompt's examples, and the query token triggers a mathematical update step equivalent to a gradient update.
This implicit optimization loop is highly sophisticated. Michael Eli Sander et al. showed in How do Transformers Perform In-Context Autoregressive Learning ? that transformers execute a two-step autoregressive algorithm: they first estimate the underlying rule of a sequence (forming a "context matrix") and then apply that rule directly to the query token.
Furthermore, this capability extends beyond simple regression. Spencer Frei et al. proved in Trained Transformer Classifiers Generalize and Exhibit Benign Overfitting In-Context that linear transformers can implement preconditioned gradient descent for classification tasks. They even demonstrate "benign overfitting" in-context, meaning the model can memorize noisy labels within a prompt while still maintaining near-optimal prediction boundaries on new test points.
This internal optimization engine can even spontaneously discover reinforcement learning algorithms. As demonstrated by Jiuqi Wang et al. in Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning, when a transformer is trained on sequence data generated by policy evaluations, it naturally implements classic Temporal Difference (TD) updates within its forward pass to evaluate new, unseen tasks.
The Circuit Architecture of Task Execution
If the forward pass is executing an algorithm, how is this "program" physically represented within the model's activations? Mechanistic interpretability tools have revealed that the model compresses the prompt's examples into highly localized, functional representations.
One primary mechanism is the creation of a task vector. Mor Geva et al. discovered in In-Context Learning Creates Task Vectors that transformers compress the multi-shot demonstrations provided in a prompt into a single, compact vector residing in the intermediate layers. By extracting this vector and patching it directly into a query-only forward pass, the model can successfully execute the task without ever directly attending to the original examples.
This process occurs at a specific point in the model's depth. In Where does In-context Learning Happen in Large Language Models?, Suzanna Sia mapped the layer-wise progression of ICL, showing a clear structural transition: early layers focus on analyzing and parsing the prompt examples, while middle-to-late layers switch entirely to executing the recognized task.
[ Input Prompt ] ---> [ Early Layers: Task Detection ] ---> [ Middle Layers: Task Vector Formed ] ---> [ Late Layers: Task Execution ]
To map these causal pathways at the single-neuron level, Dmitrii Kharlapenko et al. applied Sparse Autoencoders (SAEs) in Scaling Sparse Feature Circuits For Studying In-Context Learning. They isolated a precise two-stage circuit consisting of task-detection features (which activate when a pattern is recognized in the prompt) and task-execution features (which causally drive the final token prediction). This matches findings by Chenghao Sun et al. in Interpret and Improve In-Context Learning via the Lens of Input-Label Mappings, which showed that input-label mappings are stored as human-interpretable principal components in the middle layers; modifying just a tiny fraction of the attention heads utilizing these components can dramatically alter the model's ICL performance.
Training Dynamics and Circuit Emergence
These algorithmic circuits do not develop smoothly during a model's pretraining phase. Instead, they emerge through sharp, non-linear phase transitions.
Freya Behrens et al. analyzed this phenomenon mathematically in A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention. They proved that as a model encounters more training data, it undergoes a sudden mathematical transition, shifting from simple positional pattern matching to complex semantic understanding.
This evolution typically follows a multi-phase trajectory. Early in training, models behave like simple, "lazy" n-gram statistics matchers. In Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence, Gouki Minegishi et al. identified a distinct three-phase learning process. The model moves sequentially from basic memorization to task identification, culminating in the emergence of complex meta-learning circuits.
A critical milestone in this evolution is the formation of specialized attention structures. While basic induction heads copy exact text patterns, Jie Ren et al. identified the emergence of Semantic Induction Heads in Identifying Semantic Induction Heads to Understand In-Context Learning. These advanced circuits do not just copy tokens; they actively recall semantically related concepts (e.g., mapping "pen" to "writing"), serving as the primary mechanism that allows models to generalize to novel task instructions.
Architectural Universality
Is this capacity to build internal algorithms an exclusive property of the Transformer's attention mechanism? Evidence suggests it is actually a universal property of sequence models optimized via next-token prediction.
In Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability, Ivan Lee benchmarked 13 different architectures—including RNNs, CNNs, and State Space Models (SSMs)—and discovered that nearly all of them spontaneously develop ICL capabilities when trained properly. In fact, attention-free architectures like Mamba and Hyena occasionally outperform traditional Transformers on specific contextual tracking tasks.
This architectural flexibility was confirmed by Riccardo Grazzi et al. in Is Mamba Capable of In-Context Learning?, demonstrating that state-space models achieve comparable ICL performance to transformers across both synthetic function approximation and complex natural language processing tasks.
As argued by Andrew Gordon Wilson in Position: Deep Learning is Not So Mysterious or Different, this universality points to the presence of shared "soft inductive biases" across deep learning models. When forced to optimize for next-token prediction over diverse sequence data, highly varied architectures converge on the same underlying solution: constructing flexible, internal hypothesis spaces that act as general-purpose algorithmic interpreters.