Prompting Large Language Models for Zero Shot Inference

980 concepts806 papers28 proposals

Traditional machine learning relies on weight updates to adapt a model to a new task. In contrast, prompt-based inference treats a frozen, pre-trained Large Language Model (LLM) as a general-purpose inference engine. By altering the input text—the prompt—we change the conditional probability distribution of the next token.

Instead of teaching the model new features, prompting acts as a steering mechanism that navigates the model's existing latent space. It restricts the model's vast generative capacity to a specific subspace that aligns with the target task. Understanding why this works requires looking at how prompts activate latent world models, align task structures, and manage the computational burden of self-attention.

Activating Latent World Models

During massive pre-training, LLMs capture deep statistical regularities about language, human behavior, and domain-specific logic. They effectively construct internal "world models." Zero-shot prompting works because a well-crafted instruction can trigger these latent representations without requiring task-specific training data.

For example, in human-robot interaction, Bowen Zhang 0005 demonstrates that converting interaction scenarios into text prompts allows an LLM to act as a zero-shot human model, accurately predicting complex mental states like trust. The prompt does not teach the model empathy; it simply uncovers the social regularities already embedded in the pre-training data.

This capability extends to abstract mathematical and optimization landscapes. Hao Hao et al. demonstrate that LLMs can serve as zero-shot surrogate models for black-box optimization by including historical evaluation data directly in the prompt. Rather than computing gradients explicitly, the model interpolates within its pre-trained understanding of numerical trends and optimization principles to approximate unknown objective functions.

Lowering Entropy through Structural Alignment

An open-ended prompt forces a model to search an incredibly wide output space, which frequently leads to formatting errors, irrelevant answers, or hallucinations. To mitigate this, effective prompting strategies align the target task with structural formats that the model frequently encountered during pre-training, such as multiple-choice questions or structured reading comprehension.

By constraining the output space, we drastically lower the entropy of the prediction. For instance, Young Min Cho et al. frame unsupervised entity linking as a multiple-choice reading comprehension task. Providing a discrete list of candidate entries allows the prompt to leverage the model's comparative reasoning capabilities rather than forcing it to generate complex knowledge-base identifiers from scratch.

Similarly, Jinggui Liang et al. apply this comparative mechanism to unsupervised category discovery, transforming an abstract clustering problem into a series of pairwise exemplar comparisons. When the input format matches a highly familiar structure—such as a script format, as explored by Rudolf Rosa et al.—the model's internal representations naturally enforce the desired structural constraints without any parameter modifications.

Reducing Compositional Burden via Decomposition

Complex reasoning tasks often fail in a single zero-shot forward pass. This happens because a transformer must simultaneously parse the input, plan a multi-step solution, and format the final output within a fixed context window. Token-level and structural decomposition strategies work by distributing this compositional burden across multiple sequential inferences.

On a micro scale, Bolei Ma et al. introduce token-level prompt decomposition for sequence labeling. Instead of asking the model to label an entire sentence at once, the framework prompts the model for each token individually. This allows the self-attention mechanism to focus its entire computational capacity on local context, minimizing the interference of distant tokens.

On a macro scale, Qiangyu Tan et al. decompose complex, multi-class taxonomies into hierarchical reasoning chains. By breaking a massive classification problem into a tree of sequential, binary choices, the prompt ensures that the model's intermediate hidden states remain tightly bounded and relevant to a single, well-defined decision at a time.

Steering Attention with In-Context Contextualization

Why does adding a few examples (few-shot prompting) or forcing the model to generate its own context alter its output so dramatically? Mechanistically, the tokens provided in a prompt modify the key and value matrices ($K$ and $V$) in the transformer's self-attention layers. This shifts the activation trajectories of subsequent tokens.

Jiacheng Liu et al. leverage this via "Generated Knowledge Prompting." By instructing the model to write down what it knows about a topic before answering a question, the self-generated text serves as an explicit, dynamic memory buffer. When the model eventually generates the final answer, its attention mechanism attends to this newly created context, steering the conditional probability distribution away from generic responses and toward a highly accurate, specialized subspace.

To scale this benefit without relying on human engineering, KaShun SHUM et al. show that LLMs can automatically generate their own reasoning paths using zero-shot instructions. This process creates high-quality exemplars that are automatically tailored to the target task, optimizing the in-context attention weights for subsequent few-shot inferences.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_zero_shot_prompting_few_shot_prompting_6fdbb9bc,
  title = {zero-shot prompting (few-shot prompting)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/zero-shot-prompting-few-shot-prompting/rdr_6fdbb9bc5ed64225916f3d1efcb8d135},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_zero_shot_prompting_few_shot_prompting_6fdbb9bc_v1,
  title = {zero-shot prompting (few-shot prompting)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/zero-shot-prompting-few-shot-prompting/rdr_6fdbb9bc5ed64225916f3d1efcb8d135/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake