When an AI model evaluates multiple options—such as multiple-choice answers, entity labels, or dialogue responses—the traditional approach relies on iteration. A model typically runs $K$ separate forward passes to score $K$ candidates against a shared context. Similarly, when executing complex reasoning or planning, standard agentic workflows rely on multi-turn loops: generating a thought, acting or retrieving information, and waiting for a response before generating the next step.
While intuitive, this iterative design introduces massive computational redundancy. In candidate evaluation, the shared context is re-encoded $K$ times. In multi-turn reasoning, the system incurs significant latency from repeated API calls, sequential kernel launches, and context-switching overhead.
A powerful paradigm shift has emerged to solve this: single-pass multi-candidate encoding and structured deliberation. Instead of breaking a task into sequential, isolated pieces, these methods consolidate candidates, reasoning chains, or multiple perspectives into a single transformer forward pass or a single continuous decoding stream. By doing so, they eliminate redundant computations and allow the model's self-attention mechanism to perform joint, cross-candidate comparison natively.
The Mechanics of Context Sharing
To understand why this approach works, consider the task of response selection in dialogue systems. If you have a long conversation history (the context) and ten potential replies (the candidates), a standard cross-encoder evaluates each pair independently.
Linhao Zhang et al. challenge this in Do It Once: An Embarrassingly Simple Joint Matching Approach to Response Selection by introducing Joint Matching. Instead of ten separate passes, they bundle all candidates into a single sequence following the context. Because the shared conversation history is encoded exactly once, the model avoids redundant computations across the batch, slashing training and inference times.
Traditional Multi-Pass Evaluation:
Pass 1: [Context] + [Candidate 1] -> Score 1
Pass 2: [Context] + [Candidate 2] -> Score 2
...
Pass K: [Context] + [Candidate K] -> Score K
Single-Pass Joint Evaluation:
Single Pass: [Context] + [Candidate 1] + [Candidate 2] + ...
+ [Candidate K] -> All Scores
This concept scales across various NLP domains:
- Dialogue Systems: Chiyu Song et al. utilize a single-pass framework in Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems to achieve the deep interaction of cross-encoders with significantly lower computational overhead.
- Entity Typing: For ultra-fine entity typing, Chengyue Jiang demonstrates in Recall, Expand, and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing that concatenating multiple label candidates with a single context sequence drastically improves inference speed while maintaining high accuracy.
- Question Answering: Zhanghao Hu et al. (EEE-QA: Exploring Effective and Efficient Question-Answer Representations) and Joshua Robinson et al. (Leveraging Large Language Models for Multiple Choice Question Answering) show that processing all options simultaneously boosts throughput and provides the model with the full comparative context of available choices.
- Multi-Sentence Inference: Luca Di Liello et al. (Paragraph-based Transformer Pre-training for Multi-Sentence Inference) demonstrate that the cross-candidate reasoning enabled by a single long sequence outweighs the quadratic cost of self-attention, optimizing overall inference.
Internalizing the Deliberation Loop
The same single-pass philosophy applies to complex reasoning and planning. Traditional agentic workflows use multi-turn loops where a model generates a thought, acts, and waits for an environment response. Modern single-pass architectures internalize this entire loop into a monolithic, structured output stream.
In Retrieval-Augmented Generation (RAG), Palak Jain et al. introduce From RAG to Riches: Retrieval Interlaced with Sequence Generation, where reasoning steps and evidence retrieval are interlaced into a single decoding pass. Instead of executing slow, iterative retrieval-generation cycles, the model "writes" its retrieval targets directly into the sequence, minimizing error propagation. For deterministic operations, Ashutosh Joshi et al. (REAPER: Reasoning based Retrieval Planning for Complex RAG Systems) show that generating a complete multi-step retrieval plan in a single pass achieves a 10x reduction in latency compared to reactive agents.
This single-pass structured deliberation extends to self-contained problem-solving:
- Analogical Reasoning: Michihiro Yasunaga et al. (Large Language Models as Analogical Reasoners) demonstrate that prompting an LLM to self-generate background knowledge, relevant exemplars, and the final answer in a single continuous stream yields deep reasoning without multi-turn complexity.
- Action Deliberation: Yu Xia et al. (SAND: Boosting LLM Agents with Self-Taught Action Deliberation) train agents to generate deliberation thoughts and final actions in a single forward pass, avoiding slow test-time environment rollouts.
- Multi-Agent Compression: Do Xuan Long et al. (Multi-expert Prompting Improves Reliability, Safety and Usefulness of Large Language Models) show that a single-turn structured reasoning process can simulate a panel of experts and aggregate their insights within a single context window, outperforming slow multi-turn agent debates.
The Computational Trade-off: Why It Wins
It is well known that the computational cost of standard self-attention grows quadratically ($O(N^2)$) with sequence length. If concatenating candidates or generating long reasoning chains increases the sequence length, why is it faster?
The answer lies in the elimination of redundant operations and the optimization of hardware parallelism. Consider a scenario where we evaluate $K$ candidates against a shared context. Let $C$ be the length of the context, and $M$ be the length of an individual candidate.
In a traditional multi-pass setup, the model processes $K$ separate sequences, each of length $C + M$. The total attention computation is proportional to:
$$\text{Cost}_{\text{multi-pass}} = K \times (C + M)^2 = K C^2 + 2KCM + K M^2$$
In a single-pass setup, the candidates are concatenated into one sequence of length $C + KM$. The attention computation is proportional to:
$$\text{Cost}_{\text{single-pass}} = (C + KM)^2 = C^2 + 2KCM + K^2 M^2$$
When the shared context $C$ is large and the number of candidates $K$ is moderate, the $K C^2$ term in the multi-pass approach completely dominates the computation. For example, if $C = 1000$, $M = 10$, and $K = 100$:
- Multi-pass cost: $100 \times (1000 + 10)^2 = 102,010,000$ operations.
- Single-pass cost: $(1000 + 100 \times 10)^2 = 4,000,000$ operations.
By encoding the context $C$ exactly once, the single-pass approach achieves a massive reduction in total floating-point operations (FLOPs). Furthermore, it replaces multiple sequential kernel launches with a single, highly parallelized forward pass that maximizes GPU utilization.
This efficiency is pushed to its extreme by architectures like PSRT (Yuang Qi et al.), which optimizes virtual tokens to compress deep reasoning traces into a single forward pass classification label, turning a multi-token generation task into a constant-time inference operation.
Broad Applicability Across Tasks
Beyond text retrieval and classification, this one-stage, single-pass reasoning philosophy has proven successful across diverse domains:
- Text-to-SQL: Hanchong Zhang et al. (ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought) generate both the schema-linking reasoning chain and the final SQL query in a single inference call, bypassing multi-stage pipelines.
- Knowledge Editing: Yuchen Wu et al. (Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA) combine fact verification and logical inference into a single continuous generation pass to resist distractor facts.
- Subjective Tasks: Xiaolong Wang et al. (Perspective Transition of Large Language Models for Solving Subjective Tasks) consolidate the exploration, evaluation, and selection of multiple reasoning perspectives into a single pass.
- System Evaluation: Ke Li et al. (RAG-Zeval: Enhancing RAG Responses Evaluator through End-to-End Reasoning and Ranking-Based Reinforcement Learning) integrate claim decomposition, evidence grounding, and final judgment into a single structured output stream.
- Computer Vision: In Referring Image Segmentation, Sibei Yang et al. (Bottom-Up Shift and Reasoning for Referring Image Segmentation) demonstrate that integrating explicit reasoning modules into a one-stage framework eliminates the separate object detection stage, achieving superior speed and accuracy over traditional two-stage methods.
Key Foundational Work
Paragraph-based Transformer Pre-training for Multi-Sentence Inference (2022) — contributed 2 concept elements to this direction.
Do It Once: An Embarrassingly Simple Joint Matching Approach to Response Selection (2021) — contributed 2 concept elements to this direction.
Bottom-Up Shift and Reasoning for Referring Image Segmentation (2021) — contributed 1 concept elements to this direction.