Hierarchical Coarse to Fine Retrieval and Extraction

This direction is merged into another direction: hierarchical document filtering. Its last version stays readable and citable here.

Version 1 of 172 papers85 conceptsfrozen 2026-10-04

When a human researcher looks for a specific fact in a massive library, they do not read every page of every book in sequence. Instead, they follow a hierarchy of narrowing context: they identify the right wing of the library, then the relevant shelf, then the specific book, the chapter, and finally the paragraph containing the answer.

In modern natural language processing, this "skimming before reading" intuition has been formalized into a coarse-to-fine architecture. Rather than applying a computationally expensive model to an entire corpus or a long document, these systems use a multi-stage pipeline that progressively narrows the search space while increasing the complexity of the analysis.

The Computational Funnel

The primary motivation for this research direction is the "noise vs. recall" tradeoff. As Jinhyuk Lee and colleagues note in Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering, retrieving more documents increases the chance of finding the answer but often overwhelms the "reader" model with irrelevant text.

To solve this, researchers like Xinping Zhao propose FunnelRAG, which organizes a corpus into hierarchical levels of granularity. The process acts like a funnel:

  1. Coarse Stage: Lightweight models (like BM25 or simple embeddings) filter millions of documents or clusters down to a few hundred candidates.
  2. Intermediate Stage: Mid-sized models rerank these candidates or extract specific paragraphs.
  3. Fine Stage: High-capacity transformers perform deep semantic analysis on a tiny, high-quality subset of text.

This tiered approach, also seen in the FACILE system by Fabio Ciravegna, allows systems to maintain high throughput. By reserving resource-intensive parsing for high-priority content, Swabha Swayamdipta demonstrated that neural cascades can run up to 45x faster than flat models while processing documents ten times longer Multi-Mention Learning for Reading Comprehension with Neural Cascades.

Managing Long-Context Complexity

The coarse-to-fine philosophy is particularly effective for "long-context" problems where the input exceeds the memory limits of standard models. In Coarse-to-Fine Question Answering for Long Documents, the system mimics human behavior by first identifying relevant sentences before generating an answer, speeding up processing by up to 6.7x.

This "divide-and-conquer" strategy is also applied to summarization. Xinnian Liang and Jing Li developed C2F-FAR, which groups sentences into "semantic blocks" or facets. By filtering out insignificant blocks before selecting specific sentences, the framework handles legal and government documents much faster than methods that attempt to rank every sentence simultaneously.

Similarly, the ARTEMIS framework by Keping Bi applies this to human annotation. By having judges vet content in successive stages—from paragraphs to sections to the full document—the cognitive load is reduced, allowing for the processing of 1,300-word documents in just a few minutes.

Recent work has moved toward "agentic" implementations of this idea, where the model actively decides when to "zoom in." Ge Bai introduced GraphReader, which structures text into a graph of atomic facts. An agent evaluates these summaries first; it only retrieves the full-text chunk if the summary indicates high relevance. This prevents information overload and allows models with small context windows to outperform much larger models on massive datasets.

This concept of progressive disclosure is further refined in DeepXiv-SDK by Hongjin Qian. By providing agents with section outlines and metadata before full text, the system allows for "budget-aware" research, where the agent triages papers based on high-level summaries before committing tokens to a deep read.

Precision through Hierarchical Evidence

Beyond efficiency, the coarse-to-fine approach improves accuracy by providing structured "rationales." In XTQA, Jie Ma uses a coarse-to-fine algorithm to pinpoint specific sentence spans that justify an answer, increasing transparency.

In the realm of fact-checking, Wenting Zhao developed HUG, which treats evidence selection as a hierarchical set-prediction problem. By first selecting relevant documents and then specific sentences, the model manages the massive search space of potential evidence chains, allowing it to find complex multi-hop rationales without needing human-labeled training data.

Whether applied to chemical toxicity in Tox-chat or financial event extraction in DocFEE, the recurring lesson is clear: treating retrieval as a single-step matching problem is inefficient and noisy. By breaking the process into a hierarchy of increasingly granular filters, systems can achieve both the breadth of a library search and the precision of a close reading.

Papers in this version

Showing 60 of 72 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hierarchical_coarse_to_fine_retrieval_an_6ff50033,
  title = {Hierarchical Coarse to Fine Retrieval and Extraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-coarse-to-fine-retrieval-and-extraction/rdr_6ff50033019d42d68e70d9c5b905b5b9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_hierarchical_coarse_to_fine_retrieval_an_6ff50033_v1,
  title = {Hierarchical Coarse to Fine Retrieval and Extraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-coarse-to-fine-retrieval-and-extraction/rdr_6ff50033019d42d68e70d9c5b905b5b9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}