Single-Pass Multi-Candidate Encoding and Reasoning

28 concepts21 papers1 proposal

When an AI model evaluates multiple options—such as multiple-choice answers, entity labels, or dialogue responses—the traditional approach relies on iteration. A model typically runs $K$ separate forward passes to score $K$ candidates against a shared context. Similarly, when executing complex reasoning or planning, standard agentic workflows rely on multi-turn loops: generating a thought, acting or retrieving information, and waiting for a response before generating the next step.

While intuitive, this iterative design introduces massive computational redundancy. In candidate evaluation, the shared context is re-encoded $K$ times. In multi-turn reasoning, the system incurs significant latency from repeated API calls, sequential kernel launches, and context-switching overhead.

A powerful paradigm shift has emerged to solve this: single-pass multi-candidate encoding and structured deliberation. Instead of breaking a task into sequential, isolated pieces, these methods consolidate candidates, reasoning chains, or multiple perspectives into a single transformer forward pass or a single continuous decoding stream. By doing so, they eliminate redundant computations and allow the model's self-attention mechanism to perform joint, cross-candidate comparison natively.

The Mechanics of Context Sharing

To understand why this approach works, consider the task of response selection in dialogue systems. If you have a long conversation history (the context) and ten potential replies (the candidates), a standard cross-encoder evaluates each pair independently.

Linhao Zhang et al. challenge this in Do It Once: An Embarrassingly Simple Joint Matching Approach to Response Selection by introducing Joint Matching. Instead of ten separate passes, they bundle all candidates into a single sequence following the context. Because the shared conversation history is encoded exactly once, the model avoids redundant computations across the batch, slashing training and inference times.

Traditional Multi-Pass Evaluation:
Pass 1: [Context] + [Candidate 1] -> Score 1
Pass 2: [Context] + [Candidate 2] -> Score 2
...
Pass K: [Context] + [Candidate K] -> Score K

Single-Pass Joint Evaluation:
Single Pass: [Context] + [Candidate 1] + [Candidate 2] + ...

+ [Candidate K] -> All Scores

This concept scales across various NLP domains:

Internalizing the Deliberation Loop

The same single-pass philosophy applies to complex reasoning and planning. Traditional agentic workflows use multi-turn loops where a model generates a thought, acts, and waits for an environment response. Modern single-pass architectures internalize this entire loop into a monolithic, structured output stream.

In Retrieval-Augmented Generation (RAG), Palak Jain et al. introduce From RAG to Riches: Retrieval Interlaced with Sequence Generation, where reasoning steps and evidence retrieval are interlaced into a single decoding pass. Instead of executing slow, iterative retrieval-generation cycles, the model "writes" its retrieval targets directly into the sequence, minimizing error propagation. For deterministic operations, Ashutosh Joshi et al. (REAPER: Reasoning based Retrieval Planning for Complex RAG Systems) show that generating a complete multi-step retrieval plan in a single pass achieves a 10x reduction in latency compared to reactive agents.

This single-pass structured deliberation extends to self-contained problem-solving:

The Computational Trade-off: Why It Wins

It is well known that the computational cost of standard self-attention grows quadratically ($O(N^2)$) with sequence length. If concatenating candidates or generating long reasoning chains increases the sequence length, why is it faster?

The answer lies in the elimination of redundant operations and the optimization of hardware parallelism. Consider a scenario where we evaluate $K$ candidates against a shared context. Let $C$ be the length of the context, and $M$ be the length of an individual candidate.

In a traditional multi-pass setup, the model processes $K$ separate sequences, each of length $C + M$. The total attention computation is proportional to:

$$\text{Cost}_{\text{multi-pass}} = K \times (C + M)^2 = K C^2 + 2KCM + K M^2$$

In a single-pass setup, the candidates are concatenated into one sequence of length $C + KM$. The attention computation is proportional to:

$$\text{Cost}_{\text{single-pass}} = (C + KM)^2 = C^2 + 2KCM + K^2 M^2$$

When the shared context $C$ is large and the number of candidates $K$ is moderate, the $K C^2$ term in the multi-pass approach completely dominates the computation. For example, if $C = 1000$, $M = 10$, and $K = 100$:

  • Multi-pass cost: $100 \times (1000 + 10)^2 = 102,010,000$ operations.
  • Single-pass cost: $(1000 + 100 \times 10)^2 = 4,000,000$ operations.

By encoding the context $C$ exactly once, the single-pass approach achieves a massive reduction in total floating-point operations (FLOPs). Furthermore, it replaces multiple sequential kernel launches with a single, highly parallelized forward pass that maximizes GPU utilization.

This efficiency is pushed to its extreme by architectures like PSRT (Yuang Qi et al.), which optimizes virtual tokens to compress deep reasoning traces into a single forward pass classification label, turning a multi-token generation task into a constant-time inference operation.

Broad Applicability Across Tasks

Beyond text retrieval and classification, this one-stage, single-pass reasoning philosophy has proven successful across diverse domains:

Key Foundational Work

Paragraph-based Transformer Pre-training for Multi-Sentence Inference (2022) — contributed 2 concept elements to this direction.
Do It Once: An Embarrassingly Simple Joint Matching Approach to Response Selection (2021) — contributed 2 concept elements to this direction.
Bottom-Up Shift and Reasoning for Referring Image Segmentation (2021) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_single_pass_multi_candidate_encoding_3a2db55c,
  title = {single-pass multi-candidate encoding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/single-pass-multi-candidate-encoding/rdr_3a2db55c0d934ad9a57018e7926b6890},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_single_pass_multi_candidate_encoding_3a2db55c_v1,
  title = {single-pass multi-candidate encoding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/single-pass-multi-candidate-encoding/rdr_3a2db55c0d934ad9a57018e7926b6890/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake