Introduction
In-Context Learning (ICL) has revolutionized how we interact with Large Language Models (LLMs), allowing them to perform new tasks using only a few examples provided in the prompt. However, the quality of these demonstrations is highly variable. Some examples are redundant—containing information the model already internalized during pre-training—while others are simply noisy or irrelevant to the user's specific query. Traditional selection methods, such as BM25 or cosine similarity, often fail because they focus solely on surface-level text similarity rather than the model's internal state.
This research introduces a novel perspective: multi-layer attention acts as an amplifier. As information passes through the Transformer's layers, the model progressively prioritizes useful demonstrations and filters out noise, ensuring that the final output is driven by the most relevant context.
Theoretical Framework: The Amplifier Theorem
The authors formalize this intuition using a theoretical framework based on Linear Self-Attention (LSA) and Gradient Flow. By defining the effectiveness of a demonstration as the magnitude of its gradient flow toward the final output, they provide a mathematical basis for why certain examples drive better performance.
The core contribution is the Amplifier Theorem, which proves that in a multi-layer Transformer, the disparity between effective and ineffective demonstrations is magnified as data moves from the input layer toward the output. Essentially, the deeper layers of the model act as a lens, focusing on the most relevant and informative signals while suppressing the "redundancy trap"—information the model already knows.
GRADS: Gradient-flow-based Selection
Building on this theory, the researchers propose GRADS (Gradient-flow-based Demonstration Selection). Unlike similarity-based methods, GRADS selects examples by calculating the magnitude of their gradient flow relative to a specific query. To ensure practical utility, the method uses pre-computed demonstration encodings, making the selection process nearly as fast as standard 1-shot inference.
Empirical Results and Discussion
The effectiveness of GRADS was validated across five benchmarks (GSM8K, MATH, ARC-Challenge, MMLU-Pro, and Amazon Review) using four state-of-the-art models: Llama2-7b, Llama3.1-8b, Llama-R1-8b, and Qwen3-8b.
Key Findings
- Performance Gains: GRADS achieved a 6.8% relative improvement over strong baselines like MMR and MoD. The boost was particularly pronounced in complex reasoning tasks (MATH and MMLU-Pro), where selecting the right knowledge is critical.
- Validation of the Amplifier Effect: By tracking gradient flow across layers, the authors empirically confirmed their theorem. As shown in the figure below, the ratio of useful to useless signal (represented by the green line) consistently increases as information moves deeper into the model.
- Relevance vs. Knowledge: The research highlights that effectiveness is a dual-factor equation: how relevant a demo is to the query, and how much new information it provides. GRADS is the first method to explicitly account for both by leveraging the model's internal gradients.
Conclusion and Future Work
This study shifts the focus of demonstration selection from external similarity to internal model dynamics. While the current proofs rely on Linear Self-Attention, the empirical results across various modern architectures suggest the "amplifier" effect is a robust property of multi-layer Transformers. Future research will explore how this effect scales with multi-shot prompts and how non-linearities like Softmax attention influence the amplification process.
Summary updated