Cascaded Text Selection for Efficient Document Processing

185 concepts145 papers3 proposals

When dealing with massive corpora or long-form documents, natural language processing models face a fundamental tension between computational complexity and context length. High-capacity models—such as deep Transformers, dense reading comprehension networks, and Large Language Models (LLMs)—excel at fine-grained semantic analysis and precise information extraction. However, their computational cost often scales quadratically with input length, making it prohibitively expensive to run them over thousands of pages. Furthermore, feeding large volumes of raw text directly into a deep reader introduces substantial "noise," exposing the model to irrelevant distractors that can degrade extraction accuracy.

To resolve this tension, researchers use a coarse-to-fine cascaded selection strategy. Instead of processing an entire text collection with a single heavy model, the data is passed through a funnel of increasingly precise, increasingly expensive filters. This architecture mimics how humans navigate information: skimming a library catalog to find a book, scanning a chapter to locate a relevant paragraph, and finally reading a few sentences closely to extract an answer.

[ Massive Corpus / Long Document ]   --> High Recall / Low Cost (e.g., BM25, Heuristics)
               |
               v
     [ Relevant Paragraphs ]         --> Balanced Filter (e.g., Lightweight Classifiers)
               |
               v
       [ Target Sentences ]          --> High Precision / High Cost (e.g., Deep Readers, LLMs)
               |
               v
       [ Exact Answer/Span ]

The Funnel Architecture: From Global to Local

The core mechanism of cascaded filtering is the progressive pruning of the search space across a structural hierarchy: from documents, to paragraphs or blocks, to sentences, and finally to exact text spans.

1. Document-Level Filtering

At the mouth of the funnel, the goal is to discard entirely irrelevant texts with minimal computational effort. For instance, Jaeyoung Lee et al. demonstrate that filtering large corporate sustainability reports for domain relevance before executing semantic searches significantly cuts down computational overhead. In open-domain question answering, Arivazhagan et al. show that selecting parent documents before isolating individual passages prevents the model from retrieving text segments that look relevant in isolation but lack the necessary context provided by the surrounding document.

2. Paragraph and Block-Level Filtering

Once a relevant document is identified, it is broken down into intermediate chunks to isolate the general neighborhood of the target information. Lee et al. introduced a dedicated paragraph ranker between initial document retrieval and final answer extraction. This intermediate stage filters out noisy paragraphs, drastically boosting answer recall without overwhelming the subsequent reader. Similarly, for long-document summarization, Liang et al. group text into "semantic blocks" to eliminate redundant sub-topics before performing sentence-level extraction, speeding up processing by up to 28x compared to flat ranking methods.

3. Sentence and Span-Level Refinement

The final, most computationally intensive model operates only on a highly concentrated subset of text. Choi et al. utilize a lightweight model to filter long documents into a sparse subset of relevant sentences before a sequence-to-sequence model generates an answer, achieving up to a 6.7x speedup. In dialogue systems, Zhang et al. apply a coarse passage retriever followed by a fine-grained span extractor to ensure the generative model receives only the exact information needed to construct a response.

Why It Works: Asymmetric Optimization

The success of cascaded filtering hinges on a deliberate asymmetry between the processing stages: early stages optimize for high recall at low cost, while later stages optimize for high precision at high cost.

Computational Efficiency via Early Pruning

Early layers use low-cost operations—such as keyword heuristics, BM25, or simple feed-forward networks—to ensure that the correct information is retained while the bulk of the text is discarded. Swayamdipta et al. design a "neural cascade" using low-cost feed-forward layers to filter out noise before applying expensive attention mechanisms. This design allows for massive parallelization and enables the system to process documents several times longer than traditional sequential models without a linear increase in latency.

By the time the data reaches the heavy semantic model, the volume of text has shrunk by orders of magnitude. Yan et al. leverage this by progressively discarding irrelevant documents and paragraphs using lightweight classifiers, focusing intensive multi-task deep learning resources solely on high-probability segments.

Noise Mitigation and Grounding

Beyond saving compute, filtering fundamentally alters the data distribution seen by the final model. Feeding a deep reader an entire document full of boilerplate text or unrelated sub-topics increases the risk of false positives. As Longxuan Ma notes in a survey of open-domain dialogue systems, global-to-local knowledge selection reduces background noise, allowing models to ground their responses accurately in specific details rather than getting confused by distracting context.

Evolution of the Paradigm

The concept of tiered text processing has evolved from early rule-based pipelines into modern structure-aware architectures.

  • Early Hybrid Systems: In early information extraction frameworks, such as the FACILE system described by Ciravegna et al., a two-tiered strategy used a fast initial classifier to filter documents before passing high-priority items to an intensive information extraction module, minimizing both computational and manual parsing effort.
  • Feature-Driven Cascades: As machine learning matured, pipelines began incorporating structural and temporal features. For example, Huang et al. developed a multi-stage pipeline that progressively narrowed the search space from the document level down to individual noun phrases to detect event role fillers in secondary contexts.
  • Modern Structure-Aware Agents: In the era of LLMs and deep research agents, this paradigm manifests as structure-aware filtering. Instead of downloading and reading full webpages, modern workflows inspect document maps or structural layouts first, fetching only specific relevant segments. This targeted approach preserves downstream model performance while drastically cutting token consumption and API costs.

Key Foundational Work

FACILE: Classifying Texts Integrating Pattern Matching and Information Extraction (1999) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hierarchical_document_filtering_19c03faa,
  title = {hierarchical document filtering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-document-filtering/rdr_19c03faa88404fd283b87d7cd114e10e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_hierarchical_document_filtering_19c03faa_v1,
  title = {hierarchical document filtering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-document-filtering/rdr_19c03faa88404fd283b87d7cd114e10e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake