Unifying Retrieval and Generation in Single Transformers

20 concepts16 papers

Traditional knowledge-intensive natural language processing systems—such as those built for open-domain question answering (QA) or retrieval-augmented generation (RAG)—rely on a cascaded pipeline. In these architectures, a dense retriever first searches a massive corpus to find relevant documents, an optional reranker filters them, and a reader or generator model processes the text to synthesize the final answer.

While modular, this multi-stage approach suffers from a fundamental bottleneck: the components cannot communicate fluidly. This separation introduces several systemic flaws:

  • Error Propagation: If the retriever fails to locate the correct document, the reader has no way to recover, and the error propagates downstream.
  • Disjoint Representation Spaces: The retriever and generator operate in isolated parameter spaces. The retriever cannot leverage the rich contextual features developed by the generator, and the generator cannot influence what the retriever deems "relevant."
  • Resource Inefficiency: Running multiple large, independent models creates a massive computational and storage footprint.

To overcome these limitations, a major research direction focuses on collapsing these multi-stage pipelines into a single, unified Transformer architecture. By treating retrieval, reranking, and generation as internal operations within a single model, these architectures enable end-to-end gradient flow and mutual knowledge transfer.


Retrieval as an Internal Attention Mechanism

The core intuition behind single-model architectures is that finding information and reading information do not require separate models; instead, retrieval can be reimagined as an internal attention mechanism over a corpus.

Cascaded Pipeline:
[Query] ──> (Separate Retriever Model) ──> [Documents] ──> (Separate Generator Model) ──> [Answer]

Unified Architecture:
[Query] ──> ( Single Transformer: Internal Attention / Shared Layers ) ──> [Answer]

By framing retrieval this way, the model can optimize document selection and answer generation simultaneously. Lee et al. demonstrate this in their YONO (You Only Need One) framework, detailed in You Only Need One Model for Open-domain Question Answering. They embed the retrieval and reranking stages directly into the encoder-decoder structure of a single Transformer, treating them as internal attention operations. This allows gradients to flow uninterrupted from the final generated answer back to the initial retrieval phase.

Similarly, Jiang et al. in Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer show that a specific layer of attention within a single Transformer can be dedicated to computing document relevance. Because the reader and retriever share the same representation space, the reading component directly benefits from the internal features developed during the retrieval process, matching the performance of complex multi-model pipelines.


Shared Representations and Layer Partitioning

Another way to unify these tasks is by sharing parameters across the retrieval and generation stages, allowing the model to build a holistic understanding of the data.

Rather than maintaining separate encoders, systems can process queries and documents through a single backbone. For example, Technical Question Answering across Tasks and Domains by Yu et al. introduces a framework where query-document pairs are processed through a shared transformer-based encoder. This design allows the reading component to directly inform the retrieval process, mitigating the error propagation typical of two-stage systems.

Taking parameter sharing a step further, researchers have explored partitioning the internal layers of a single model for different sub-tasks:

This structural integration enables mutual knowledge transfer. The insights the model gains while learning to locate information refine its internal representations, which in turn makes it more accurate at synthesizing answers.


Autoregressive Generative Retrieval

A particularly powerful manifestation of this single-model philosophy is generative retrieval. Instead of calculating dense vector similarities between a query and an external index, the model learns to generate unique document identifiers (DocIDs) directly as text tokens.

In CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks, Li et al. treat DocIDs as natural language tokens within a single autoregressive decoding process. This completely eliminates the need for massive external vector indices. The model seamlessly transitions from generating the DocID of a source document to generating the final answer text in a single, continuous decoding sequence.

Similarly, Jain et al. introduce 1-PAGER in 1-PAGER: One Pass Answer Generation and Evidence Retrieval. This architecture uses constrained decoding to generate a specific "search path" of keywords alongside the final answer in a single decoding pass. By integrating retrieval directly into the generation phase, it provides human-readable attribution while bypassing the training difficulties of coordinating separate retrieval and reading components.


Unifying Adjacent Pipeline Stages

The drive toward unification is not limited solely to the retriever-reader boundary; it extends to other adjacent stages of the pipeline, proving that tightly coupled architectures consistently outperform isolated silos.

Query Understanding and Encoding

Before retrieval even happens, queries often need to be rewritten or expanded. In UniRAG: Unified Query Understanding Method for Retrieval Augmented Generation, Li et al. merge query rewriting and vector encoding into a single decoder-only Large Language Model. This prevents the information loss that occurs when independent, non-communicating systems handle rewriting and embedding separately.

Retrieval and Reranking

Combining the initial search and subsequent filtering stages into one model drastically improves efficiency. Bhat et al. introduce this concept for general text search in UR2N: Unified Retriever and ReraNker, creating a single end-to-end trainable encoder-decoder model. For specialized tasks like entity linking, Bhowmik et al. (Fast and Effective Biomedical Entity Linking Using a Dual Encoder) collapse candidate retrieval and reranking into a single dual-encoder space, learning a joint representation for mentions and entities.

Conversational Search and Multimodal QA

In multi-turn interactions, keeping retrieval and generation aligned is notoriously difficult. To address this, Mo et al. developed UniConv, detailed in UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations. UniConv trains a single LLM to handle both retrieval and response generation, ensuring conversational consistency.

This joint optimization is equally effective when reasoning over structured and unstructured data (Joint Reasoning on Hybrid-knowledge sources for Task-Oriented Dialog by Contractor et al.), or when locating and reading visual information in product manuals (MPMQA: Multimodal Question Answering on Product Manuals by Zhang et al.). Across all these domains, learning to find information and learning to synthesize it are treated as complementary tasks that thrive on a shared feature space.


Extreme Efficiency and Edge Deployment

Beyond the clear architectural and performance advantages, consolidating the retrieval-generation pipeline yields massive practical benefits for deployment. Multi-model pipelines are notoriously resource-heavy, often requiring tens of gigabytes of memory to hold separate retrieval indices, rerankers, and language models.

By compressing these capabilities into a single model, the physical footprint of the system drops dramatically. In Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering, Yang et al. demonstrate that a single lightweight encoder trained via knowledge distillation can successfully perform both document retrieval and reading tasks. This optimization shrinks the storage footprint of a standard open-domain QA system by 160x (from 77.5 GB to under 500 MB), proving that unified models are not only more accurate but can also be made small enough to run entirely on edge devices.

Key Foundational Work

You Only Need One Model for Open-domain Question Answering (2022) — contributed 2 concept elements to this direction.
Fast and Effective Biomedical Entity Linking Using a Dual Encoder (2021) — contributed 2 concept elements to this direction.
Technical Question Answering across Tasks and Domains (2021) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_unified_retriever_reader_architecture_d30bfda7,
  title = {unified retriever-reader architecture},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unified-retriever-reader-architecture/rdr_d30bfda7dd01472c8b4481a1b3457509},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_unified_retriever_reader_architecture_d30bfda7_v1,
  title = {unified retriever-reader architecture},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unified-retriever-reader-architecture/rdr_d30bfda7dd01472c8b4481a1b3457509/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake