In-Context Learning (ICL) has fundamentally changed how we interact with Large Language Models (LLMs). As noted by Qingxiu Dong et al. in A Survey on In-context Learning, providing a few examples within a prompt allows an LLM to execute entirely new tasks at inference time without a single parameter update.
Yet, beneath this capability lies a profound fragility. ICL is notoriously volatile: shuffling the order of demonstrations, changing a single word, or altering the prompt formatting can cause model performance to swing violently between state-of-the-art accuracy and random guessing (Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning).
To understand why this instability occurs and how to fix it, we must look at how transformers route information through their attention layers, how prompts introduce structural biases, and how researchers are engineering solutions to stabilize this emergent behavior.
The Mechanics of Prompt Bias and Information Flow
Why does a model care so much about the specific sequence or phrasing of examples? The answer lies in how transformers process context.
Some theoretical perspectives suggest that ICL is algorithmically equivalent to implicit gradient descent, treating the context window as a space for virtual weight updates. However, Lingfeng Shen et al. (Position: Do pretrained Transformers Learn In-Context by Gradient Descent?) demonstrate a fundamental functional divergence: while batch gradient descent is mathematically invariant to sample order, ICL is highly sensitive to it. This implies that ICL's sensitivity is uniquely tied to sequential attention mechanisms rather than standard optimization pathways.
An elegant explanation for this behavior is provided by Hao Zhou et al. in Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning. They reveal that during the forward pass, label words (such as "Positive" or "Negative") act as semantic anchors.
[Example 1: Text] -> [Label 1 (Anchor)] <-\
\-- [Query Token] extracts consolidated info
[Example 2: Text] -> [Label 2 (Anchor)] <--/
In shallow layers, these anchor tokens aggregate semantic information from their preceding demonstration text. In deeper layers, the model extracts this consolidated information from the anchors to form its final prediction for the query token. If the demonstrations are poorly constructed, noisy, or arranged in an unfavorable sequence, the information flow to these anchors becomes distorted.
This structural vulnerability frequently manifests as "Template Bias"—a tendency for the model to favor certain labels based entirely on the prompt's formatting or example order rather than the underlying task logic (Towards Informative Few-Shot Prompt with Maximum Information Gain for In-Context Learning). For instance, Huan Ma et al. (Fairness-guided Few-shot Prompting for Large Language Models) observe that when given a meaningless or "null" input, a biased prompt configuration will still cause the model to heavily predict a specific label, exposing a latent unfairness in the prompt's structure.
Principled Demonstration Selection
If ICL is highly sensitive to the choice of examples, how do we find the optimal set? Moving away from random selection or simple semantic similarity, researchers have developed several frameworks to optimize what goes into the context window.
Information Theory and Compression
The authors of Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering approach prompt design through the lens of the Minimal Description Length (MDL) principle. Instead of using a static, "one-size-fits-all" prompt, they dynamically select and rank examples that maximize the model's compression efficiency and confidence for each specific input query.
Diversity over Similarity
While intuition suggests picking examples that are most similar to the query, Wenyang Xiao et al. (The Role of Diversity in In-Context Learning for Large Language Models) demonstrate that diversity within the demonstration pool is often more critical, particularly for complex reasoning and coding tasks.
To exploit this, Costas Mavromatis et al. developed algorithms like ADAICL (Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection) and CoverICL (CoverICL: Selective Annotation for In-Context Learning via Active Graph Coverage). These methods combine model uncertainty with graph-based maximum coverage strategies to select a highly informative, diverse subset of examples for annotation, maximizing performance while minimizing human effort.
Preference-Aligned Retrieval
Rather than relying on heuristics, Xiaonan Li et al. introduced frameworks like LENS (Finding Support Examples for In-Context Learning) and UDR (Unified Demonstration Retriever for In-Context Learning) to filter and retrieve optimal support examples.
Crucially, Zheng Zhang et al. (Learning to Select In-Context Demonstration Preferred by Large Language Model) treat selection as a generative preference learning problem. They show that when a model is provided with demonstrations aligned with its internal architectural preferences, its sensitivity to example ordering drops drastically, neutralizing the sequence-ordering bottleneck.
Calibration and Reweighting Mechanisms
When we cannot change or curate the examples themselves, we can stabilize ICL by calibrating how the model interprets them during inference.
Decision Boundary Calibration
Zhixiong Han introduced Prototypical Calibration (Prototypical Calibration for Few-shot Learning of Language Models), which fits a Gaussian Mixture Model to the model's outputs on unlabeled data. By adaptively shifting the decision boundary to match the model's actual output distribution, this method neutralizes the permutation sensitivity that distorts raw predictions.
Example Reweighting
Rather than treating all demonstrations as equally valid, Zhe Yang et al. (Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning) introduced Weighted In-Context Learning (WICL). By calculating a "Masked Self-Prediction" score, they dynamically assign unique weights to each demonstration, ensuring that noisy or unhelpful examples do not derail the model's internal attention routing.
Implicit Augmentation
The authors of Enhancing In-Context Learning via Implicit Demonstration Augmentation take calibration into the model's deep feature space. They mathematically prove that performing infinite feature-space augmentations on demonstrations is equivalent to a direct logit calibration ("IDA-Softmax"). This allows the model to achieve the stability of a massive ensemble of prompts without actually increasing the input context length.
Scaling Context and Cross-Modal Sensitivity
As context windows expand to millions of tokens, one might assume that adding more examples naturally dilutes sensitivity. However, scaling introduces its own structural challenges.
Zaheer Abbas (Many-Shot In-Context Learning) demonstrates that scaling to hundreds or thousands of examples allows models to override pre-training biases and achieve massive performance gains. Yet, even in this "many-shot" regime, order-dependent instability remains a persistent challenge.
In complex settings like agent-based imitation learning, Anian Ruoss et al. (LMAct: A Benchmark for In-Context Imitation Learning with Long Multimodal Demonstrations) identify a phenomenon called "in-context interference." Here, adding more long multimodal demonstrations can actively degrade performance due to extreme sensitivity to how environment states and action sequences are formatted and encoded.
To mitigate this across different modalities and tasks, researchers are shifting from sample-specific optimization to broader task-level frameworks. For instance, Yan Zhu et al. (Exploring Task-Level Optimal Prompts for Visual In-Context Learning) discovered that in visual in-context learning, a single optimized set of prompt examples can effectively serve an entire task. This shifts the focus from expensive, query-by-query prompt searches to stable, task-level configurations.
Key Foundational Work
OpenICL: An Open-Source Framework for In-context Learning (2023) — contributed 2 concept elements to this direction.
Measuring Pointwise \mathcalV-Usable Information In-Context-ly (2023) — contributed 2 concept elements to this direction.
What Makes Good Examples for Visual In-Context Learning? (2023) — contributed 2 concept elements to this direction.