Multi-Phase Circuit Emergence in Transformer Training

13 concepts7 papers1 proposal

When observing a Transformer during training, the acquisition of high-level capabilities often looks like a series of sudden, discrete leaps. A model might struggle with a specific grammatical rule, logical structure, or in-context learning task for thousands of iterations, showing near-zero competence, only to master it rapidly over a brief window of training steps.

As shown by Aryaman Arora et al. in CausalGym: Benchmarking causal interpretability methods on linguistic tasks, internal circuits responsible for complex syntax emerge in these abrupt, non-linear stages rather than through gradual improvement. Similarly, Isabelle Lee et al. demonstrated in (Pre-)training Dynamics: Scaling Generalization with First-Order Logic that generalization on formal logic tasks follows a distinct, staircase-like progression.

Why does learning manifest as a staircase rather than a smooth hill? Mechanistic analysis reveals that these sudden macro-level phase transitions are an illusion. Beneath the surface, they are driven by highly structured, multi-phase developmental progressions where individual subcircuits and attention heads must smoothly co-evolve, compete, and eventually cooperate before a capability can unlock.

The Illusion of the Quantum Leap

To understand why capabilities appear suddenly, we must look at how subcircuits interact during optimization. In What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation, Aaditya K Singh et al. introduced an "artificial optogenetics" framework to causally clamp internal activations at various points during training. This allowed them to isolate the constituent parts of an induction head—the core circuit responsible for in-context learning (ICL).

The authors discovered that the sudden appearance of induction capabilities is not a single, spontaneous phase change. Instead, it is driven by three distinct subcircuits that each grow smoothly and exponentially. Because the final macro-circuit requires all three components to function, the model's performance remains flat until the final subcircuit crosses a critical threshold. Once it does, the subcircuits connect, creating the illusion of a sudden capability spike.

Subcircuit A (Smooth) ───┐
Subcircuit B (Smooth) ───┼─► Critical Threshold ─► Sudden Macro-Capability
Subcircuit C (Smooth) ───┘

From Competition to Cooperation

This underlying multi-phase dynamic is closely tied to how individual attention heads coordinate over time. When a network is presented with a sequential task, the attention heads do not immediately divide labor.

As observed by Oğuz Kaan Yüksel et al. in Incremental Learning of Sparse Attention Patterns in Transformers, attention heads initially enter a competitive phase. During this period, multiple heads vie for the same dominant, easily accessible information signals in the data. This redundancy allows the network to secure early, rapid accuracy gains on simple patterns, but it limits overall architectural efficiency.

Once these dominant signals are saturated, the network transitions into a cooperative phase. In a theoretical analysis of this shift, Oğuz Kaan Yüksel demonstrated in Incremental Learning of Sparse Attention Patterns in Transformers that heads begin to differentiate and specialize. A subset of heads maintains the foundational patterns already learned, while other heads are freed to compensate, shifting their focus toward sparser, more complex sequential patterns. This transition from competition to specialized labor division marks the boundary between distinct training plateaus.

The Sequential Assembly Line of In-Context Learning

When we look at specific capabilities like ICL, this transition from simple to complex patterns follows a highly predictable, step-by-step assembly line.

By mathematically modeling the training process, Tiberiu Musat et al. proved in On the Emergence of Induction Heads for In-Context Learning that induction heads form in a strict three-stage sequence:

  1. The model first learns to output general label averages.
  2. It develops the mechanism to attend to specific labels (refining its Query-Key circuit).
  3. It finally masters the precise mapping between items and their preceding tokens (refining its Output-Value circuit).

Mingze Wang et al. provided a complementary four-phase framing in How Transformers Get Rich: Approximation and Dynamics Analysis, tracking how the model transitions from a "lazy" partial n-gram model, through a prolonged learning plateau, into the rapid emergence of "rich" induction heads, and finally to convergence.

Crucially, simpler structural primitives always precede complex symbolic associations. For instance, Sultan Daniels et al. demonstrated in Decomposing Prediction Mechanisms for In-context Recall that a transformer's ability to simply continue an ongoing in-context pattern develops significantly earlier in training than its capacity to perform true associative recall based on abstract symbolic labels. Pattern recognition acts as the developmental scaffold for symbolic reasoning.

Scaling Up to Meta-Learning Circuits

When the training task is scaled up to require true meta-learning—where a model must infer an abstract task rule from context examples rather than just copying tokens—this multi-phase circuit emergence becomes even more pronounced.

In In-Context Meta Learning Induces Multi-Phase Circuit Emergence, Gouki Minegishi et al. tracked the internal structural evolution of models during meta-learning tasks. They identified three macro-circuits that emerge sequentially, each corresponding to a distinct accuracy plateau:

  • Non-Context Circuit (NCC): Early in training, the model ignores the context entirely. It relies purely on static, weight-based priors embedded in its parameters to make educated guesses.
  • Semi-Context Circuit (SCC): The model transitions to a label-set heuristic. It becomes aware of the context labels but cannot yet bind them to their corresponding inputs.
  • Full-Context Circuit (FCC): Finally, the model achieves full input-label binding, enabling true task inference and sophisticated in-context meta-learning.

As Gouki Minegishi notes in Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence, this progression proves that meta-learning is not achieved via a single monolithic leap, but rather through a highly ordered, multi-stage structural evolution.

A Systems-Level Perspective

This staged reconfiguration is not unique to Transformers; it reflects a fundamental systems-level property of neural network optimization. By analyzing network structures through the lens of network neuroscience, James M. Shine et al. in Nonlinear reconfiguration of network edges, topology and information content during an artificial learning task mapped out three global topological phases that mirror these findings. Networks begin with rapid accuracy gains via minimal structural changes, transition into a highly modular state that drives specialized performance, and finally reduce modularity to refine pattern separation.

Understanding these learning dynamics shifts our view of neural networks from static mathematical functions to dynamic, evolving ecosystems. Capabilities do not just scale smoothly with compute; they are built piece by piece, phase by phase, through an intricate internal dance of competition, cooperation, and circuit assembly.

Key Foundational Work

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation (2024) — contributed 1 concept elements to this direction.
CausalGym: Benchmarking causal interpretability methods on linguistic tasks (2024) — contributed 1 concept elements to this direction.
Nonlinear reconfiguration of network edges, topology and information content during an artificial learning task (2021) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_staged_learning_dynamics_deaf8af5,
  title = {staged learning dynamics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/staged-learning-dynamics/rdr_deaf8af542c14c689aed3e6ef12c3644},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_staged_learning_dynamics_deaf8af5_v1,
  title = {staged learning dynamics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/staged-learning-dynamics/rdr_deaf8af542c14c689aed3e6ef12c3644/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake