No article text was recorded for this version.
mechanistic interpretability of in-context learning
Version 1 of 1166 papers174 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Implicit Statistical Inference in Transformers: Approximating Likelihood-Ratio Tests In-Context
- Blog: Transformers as Stochastic Optimizers
- Blog: On the Role of Unstructured Training Data in Transformers' In-Context Learning Capabilities
- Blog: Exact Learning Dynamics of In-Context Learning in Linear Transformers and Its Application to Non-Linear Transformers
- Blog: What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning
- Blog: Task Vectors in In-Context Learning: Emergence, Formation, and Benefits
- Blog: Data Distributional Properties Drive Emergent In-Context Learning in Transformers
- Blog: Trained Transformer Classifiers Generalize and Exhibit Benign Overfitting In-Context
- Blog: Training Dynamics of In-Context Learning in Linear Attention
- Blog: Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification
- Blog: Latent Concept Disentanglement in Transformer-based Language Models
- Blog: Understanding Emergent In-Context Learning from a Kernel Regression Perspective
- Blog: Distributed Rule Vectors is A Key Mechanism in Large Language Models' In-Context Learning
- Blog: Transformers as Multi-Task Feature Selectors: Generalization Analysis of In-Context Learning
- Blog: Transformer learns the cross-task prior and regularization for in-context learning
- Blog: Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning
- Blog: Understanding In-context Learning of Addition via Activation Subspaces
- Blog: Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection
- Blog: How Do Nonlinear Transformers Acquire Generalization-Guaranteed CoT Ability?
- Blog: How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding
- Blog: Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction
- Blog: Identifying Semantic Induction Heads to Understand In-Context Learning
- Blog: Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers
- Blog: Asymptotic theory of in-context learning by linear attention
- Blog: How recurrent networks implement contextual processing in sentiment analysis
- Blog: In-context Learning on Function Classes Unveiled for Transformers
- Blog: Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking
- Blog: Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Models
- Blog: Inducing Induction in Llama via Linear Probe Interventions
- Blog: CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations
- Blog: Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression
- Blog: Dissecting In-Context Learning of Translations
- Blog: Comparing the learning dynamics of in-context learning and fine-tuning in language models
- Blog: Schema-learning and rebinding as mechanisms of in-context learning and emergence
- Blog: From Compression to Expansion: A Layerwise Analysis of In-Context Learning
- Blog: Mechanics of Next Token Prediction with Self-Attention
- Blog: An Explanation of In-context Learning as Implicit Bayesian Inference
- Blog: Explicit Knowledge Factorization Meets In-Context Learning: What Do We Gain?
- Blog: How Do Transformers Model Physics? Investigating the Simple Harmonic Oscillator
- Blog: Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models
- Blog: Towards Understanding the Universality of Transformers for Next-Token Prediction
- Blog: Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning
- Blog: Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs
- Blog: Language model acceptability judgements are not always robust to context
- Blog: A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Tasks
- Blog: Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
- Blog: One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention
- Blog: Can Transformers Learn Tasks of Varying Complexity In-context?
- Blog: How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learning
- Blog: Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection
- Blog: Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
- Blog: Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
- Blog: ICLR: In-Context Learning of Representations
- Blog: Spin-glass model of in-context learning
- Blog: How do Transformers Perform In-Context Autoregressive Learning ?
- Blog: In-Context Learning in Large Language Models: A Neuroscience-inspired Analysis of Representations
- Blog: In-Context Operator Learning for Differential Equation Problems
- Blog: How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations
- Blog: Learning Linear Regression with Low-Rank Tasks In-Context
- Blog: Scaling Sparse Feature Circuits For Studying In-Context Learning
Showing 60 of 166 papers.
History & citation
Current · Permanent link
- The Algorithmic Mechanics of In-Context Learning merged into this direction (54% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_mechanistic_interpretability_of_in_conte_472ce5c0,
title = {mechanistic interpretability of in-context learning},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/mechanistic-interpretability-of-in-context-learning/rdr_472ce5c0479e462e852e09397c6536ce},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_mechanistic_interpretability_of_in_conte_472ce5c0_v1,
title = {mechanistic interpretability of in-context learning},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/mechanistic-interpretability-of-in-context-learning/rdr_472ce5c0479e462e852e09397c6536ce/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}