tRAG: Term-level Retrieval-Augmented Generation for Domain-Adaptive Retrieval

Dohyeon Lee, Jongyoon Kim, Jihyuk Kim, Seung-won Hwang, Joonsuk ParkNAACL 2025 · DOI

Abstract

Neural information retrieval models often struggle when deployed in specialized domains due to a lack of labeled training data. While Large Language Models (LLMs) are frequently used for Pseudo-Query Generation (PQG) to bridge this gap, they exhibit a significant "seen term bias," predominantly reusing vocabulary found within the source document. We introduce tRAG (term-level Retrieval-Augmented Generation), a framework that retrieves specific, relevant keywords from across an entire corpus to guide LLMs. By shifting from document-level to term-level retrieval, tRAG enables the generation of more diverse and realistic queries containing "unseen" terms, significantly improving zero-shot retrieval performance.

The Problem: Seen Term Bias in PQG

In neural information retrieval, models are trained to map documents to queries. Moving to a new domain (e.g., medical or financial) typically requires domain adaptation. The standard approach, Generative Pseudo Labeling (GPL), uses an LLM to generate queries for a document. However, researchers have identified a major flaw: seen term bias. LLMs tend to be uncreative, reusing terms directly from the input document rather than employing the diverse terminology real users might use (e.g., searching for "statin" to find a paper about "cholesterol drugs"). This failure to capture "out-of-document" terms results in models that are poorly adapted to actual human search behavior.

Methods: Term-level Retrieval-Augmented Generation

tRAG addresses this bias by surgically retrieving relevant keywords from the entire corpus to provide a denser, more relevant signal to the LLM than standard document-level RAG.

The tRAG architecture compared to standard LLM generation and document-level RAG.

The tRAG pipeline follows a "Generate-then-Rank" approach:

  1. Keyword Generation: The LLM generates domain-specific keywords for every document in the corpus.
  2. Term-level Retrieval: Instead of pulling in full, noisy documents, tRAG uses "centroid keywords" as the retrieval unit.
  3. Collective Verification (CoVe): To prevent hallucinations, keywords are verified against the entire corpus and ranked based on relevance to both the document and the initial query.
  4. Query Refinement: The LLM integrates the original document, a summary of retrieved documents, and verified high-recall keywords to produce a final pseudo-query containing both seen and unseen terms.

Experimental Results

The authors evaluated tRAG using the BEIR benchmark, focusing on a challenging subset (BEIR$_{OOC}$) with high ratios of out-of-corpus terms, including NFCorpus, SciFact, SciDocs, and FiQA.

Recall of Unseen Terms

A primary metric for success is the model's ability to generate terms not present in the original document. tRAG demonstrated a substantial improvement in this area, boosting the recall of "unseen" query terms by 10.6 percentage points over the standard GPL baseline.

Comparison of recall for unseen terms across different datasets and methods.

Retrieval Performance and Hallucination Control

On the BEIR benchmark, tRAG outperformed baseline LLM generation and standard RAG by 1.8%pt and 1.3%pt respectively in terms of nDCG@10. Crucially, the Collective Verification step ensured that these new terms were grounded in the domain; the hallucination rate (generating non-existent domain terms) remained minimal at approximately 1%.

Discussion and Conclusion

tRAG demonstrates that the "seen term bias" is not solved by simply scaling up LLMs; even massive models like GPT-3 require external grounding to overcome their tendency to summarize rather than expand. By focusing on term-level retrieval, tRAG bypasses the "lost-in-the-middle" noise associated with standard document-level RAG.

While tRAG introduces some computational overhead during the training/adaptation phase due to keyword verification, it does not impact inference speed at runtime. Future work may focus on optimizing few-shot example selection for keyword generation to further enhance the diversity and accuracy of the generated queries.

Summary updated

BibTeX
@inproceedings{lee2025trag,
  author = {Dohyeon Lee and Jongyoon Kim and Jihyuk Kim and Seung-won Hwang and Joonsuk Park},
  title = {{tRAG: Term-level Retrieval-Augmented Generation for Domain-Adaptive Retrieval}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.334},
}
Figure