Dynamic Retrieval for In-Context Learning

This direction is split into other directions: semantic example retrieval, similarity-based prompt retrieval. Its last version stays readable and citable here.

Version 1 of 11,400 papers1,718 conceptsfrozen 2026-10-04

Large Language Models (LLMs) are notoriously sensitive to the context provided in their prompts. In the "few-shot" paradigm, a model is given a few examples (demonstrations) to guide its behavior. However, using a static, fixed set of examples for every query is often suboptimal. If you ask a model to solve a complex calculus problem, showing it examples of basic addition doesn't help, even if those addition examples are "high quality."

The research direction of Dynamic Retrieval for In-Context Learning (ICL) replaces static prompts with a just-in-time retrieval mechanism. Instead of a one-size-fits-all prompt, the system looks at the specific user query and searches a large datastore to find the most relevant demonstrations to include in the context window.

The Intuition: Analogical Reasoning

The core reason this works is that LLMs function as powerful analogical reasoners. When a model sees a demonstration that shares a similar logical structure or domain with the current query, it can "map" the solution pattern from the example to the new problem.

Standard ICL often relies on random selection, which Man Luo and colleagues in Dr.ICL argue fails to account for the specific nuances of individual queries. By dynamically retrieving examples that are semantically or logically similar, we provide the model with a "logical blueprint" rather than just a generic instruction.

From Semantic Similarity to Functional Relevance

Early approaches, such as LM-BFF by Tianyu Gao and Adam Fisch, demonstrated that selecting examples based on semantic similarity—using dense encoders to find "nearby" text—yields massive improvements over random selection.

However, recent work has moved beyond simple similarity. The goal is now functional relevance: finding examples that actually help the model get the right answer, even if they don't look similar on the surface.

  • Trainable Retrievers: In LM-PDD, Kaijian Li and others developed a retriever that learns which demonstrations lead to the highest accuracy in the generator. This creates a feedback loop where the retriever prioritizes structural or logical guidance over topical similarity.
  • Reinforcement Learning: Xubin Wang and Jianfei Wu treat demonstration selection as a reinforcement learning problem, optimizing the selection process to maximize the LLM's final performance.
  • Outcome-Based Selection: In negotiation tasks, YUNCHENG HUA and Lizhen Qu found that selecting examples based on their historical contribution to successful outcomes is more effective than semantic matching.

Scaling Context via External Datastores

One of the primary constraints of LLMs is the context window. You cannot put 10,000 examples in a prompt. Dynamic retrieval effectively bypasses this limit.

In kNN-ICL, Ye Liu and Yao Wan treat semantic parsing as a code generation task where the model queries a massive datastore at inference time. This allows the model to access a vast library of domain-specific knowledge—such as complex, nested commands—without needing to be fine-tuned on all of them. Similarly, Chenlu Zhan and Gaoang Wang use a "shareable prompt space" in UniDCP to dynamically retrieve clinical concepts for medical vision tasks, allowing a single model to handle diverse medical datasets.

Beyond Text: Multi-modal and Structural Dynamics

The principle of dynamic selection extends to complex reasoning chains and multi-modal data.

  • Decomposition: Dheeru Dua and Sameer Singh show in Successive Prompting that for complex questions, you should select different demonstrations for each sub-step of the reasoning process.
  • Multi-modal: Bingshuai Liu introduced MM-Retrieval, which uses cross-modal retrieval to find relevant image-text pairs to guide models like GPT-4V through scientific reasoning.
  • Model Compression: Perhaps most surprisingly, Duc Hoang demonstrated that Inference-time Dynamic Prompting (IDP) can recover knowledge "lost" during model pruning or quantization. By selecting prompts based on internal attention scores, the model can be "redirected" to find displaced knowledge without expensive retraining.

Why It Matters

Dynamic retrieval transforms the LLM from a static knowledge base into a flexible reasoning engine that can adapt its "expertise" to the specific query at hand. By automating the selection of the most effective demonstrations, researchers are moving away from the "black art" of manual prompt engineering toward a systematic, data-driven architecture for machine intelligence.

Papers in this version

Showing 60 of 1,400 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dynamic_retrieval_for_in_context_learnin_b832a17e,
  title = {Dynamic Retrieval for In-Context Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-retrieval-for-in-context-learning/rdr_b832a17e2a2a427080ea7b8de868ec5c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_dynamic_retrieval_for_in_context_learnin_b832a17e_v1,
  title = {Dynamic Retrieval for In-Context Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-retrieval-for-in-context-learning/rdr_b832a17e2a2a427080ea7b8de868ec5c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}