When extracting information or training language models on web-scale text, local context can be highly deceptive. A sentence containing a name, a date, and a location might look like a biographical fact, but if it is embedded within a fictional story or a forum argument, extracting it introduces severe noise. Relying solely on token-level or sentence-level features often leads to "semantic drift," where algorithms lose focus because words share local contexts across entirely different domains.
To solve this, researchers use document-level genre and register filtering. By classifying the overarching functional type of a document—such as an FAQ, an academic paper, an opinion piece, or a biography—before performing downstream processing, systems establish a global context. This macro-level filter acts as a structural gatekeeper, dramatically reducing noise, optimizing computational efficiency, and allowing downstream algorithms to adapt their extraction strategies to the density of the text.
Resolving Local Ambiguity with Global Context
Why does document-level classification work so well when local extraction fails? The core intuition is that the document's genre dictates the rules of engagement for the text within it.
Sadamitsu et al. (2011) demonstrated this concept in the context of entity set expansion. When bootstrapping lists of entities, local context windows often cause algorithms to drift into unintended categories. By introducing document topic and genre information derived from Latent Dirichlet Allocation (LDA) as a global feature, the authors successfully differentiated entities that appeared in identical local sentence structures but belonged to distinct macro-domains.
[Unfiltered Web Crawl] ──> (Local Context: "X was born in Y") ──> High Noise / Semantic Drift
│
▼
[Genre Filter: Biography] ──> (Local Context: "X was born in Y") ──> High-Precision Fact Extraction
Similarly, Huang et al. (2011) showed that identifying a document's genre allows an information extraction system to dynamically adjust its strategy. In high-density event narratives, a system can aggressively hunt for implicit details and role fillers in secondary contexts (such as hospital transports or arrests following a bombing). However, applying this aggressive strategy to a document that only mentions an event in passing would yield massive false positives. By filtering and isolating primary narratives first, the authors maintained high precision while boosting recall.
Computational Gating in Large-Scale Harvesting
Beyond improving precision, genre filtering serves as a critical computational optimization for building large-scale datasets from raw web crawls. Processing petabytes of unorganized text with heavy token-level classifiers or Large Language Models (LLMs) is prohibitively expensive.
To build the Turku WebQA dataset, Myntti et al. (2024) implemented a resource-efficient pipeline that uses a multilingual document classifier as an initial gatekeeper. By identifying specific web registers—such as FAQs or question-answer forums—at the document level, the authors isolated high-probability text zones before applying expensive token-level classification. This structural gating prevented the system from wasting compute on irrelevant web content without sacrificing the quality of the final dataset.
A similar approach was pioneered by de Rijke et al. (2004) for open-domain question answering. When curated databases are unavailable, searching the open web introduces immense noise. By filtering web search results through a classifier trained to recognize the specific linguistic "shape" of a biography, the authors ensured that the downstream extraction mechanics only processed documents adhering to the expected factual density of the target domain.
This structural verification is equally vital for digital libraries. In maintaining the CiteSeerX engine, Wu et al. (2014) highlighted the necessity of automatically distinguishing formal academic publications from non-scholarly documents like resumes or newsletters, ensuring the integrity of autonomous citation indexing.
Curation and Composition for Language Models
Document-level register identification has evolved from a tool for filtering out noise into a mechanism for the intentional composition of pretraining data.
In an analysis of LLM pretraining data, Myntti et al. (2025) utilized a pre-trained multi-label classifier to assign linguistic registers to individual documents within a massive corpus. The authors discovered that specific registers, such as "Opinion," significantly boost model performance, whereas others, like "News," can be detrimental. This indicates that data curation must move beyond simple quality heuristics toward nuanced, genre-aware selection.
[Raw Corpus] ──> [Multi-Label Register Classifier] ──> [Opinion] ──> High LLM Utility
──> [Instruction] ──> Specialized Subsets
──> [News] ──> Downweighted
This granular control is foundational for modern open-source corpora. For instance, in the creation of the HPLT v2 dataset, Burchell et al. (2025) assigned functional register labels (e.g., "instructional" or "opinion") to documents across 193 languages. Providing this structural metadata allows researchers to extract specialized subsets for domain-specific training and conduct highly targeted evaluations of model capabilities.
The Feasibility of Macro-Level Filtering
For genre-based filtering to be viable as a first-stage gatekeeper, the underlying classifiers must be exceptionally accurate and computationally lightweight. Across various domains, literature demonstrates that document-level classifiers consistently achieve the required reliability:
- Wikipedia Semantification: The KYLIN system, developed by Weld et al. (2007), achieved an average precision of 98.5% in identifying articles belonging to specific structural classes, enabling autonomous information extraction.
- Medical Literature: The MScanner tool, created by Poulter et al. (2008), achieved ROC areas between 0.97 and 0.99 when scanning the entire MEDLINE database to separate relevant scientific citations from irrelevant ones.
- Webpage Categorization: In evidence-based healthcare systems, Zhao et al. (2010) demonstrated a Macro-averaged F1-score of 0.98 when classifying web pages into functional types like "abstract" versus "full text."
- Specialized Domains: Highly consistent performance has been replicated using simple algorithms like Naive Bayes for educational reform documents (Ma et al., 2024), keyword-based precision filters to isolate war-related content in unconventional side-channels (Moreno et al., 2024), and visual document filtering to isolate music score pages from non-score pages (Lin et al., 2024).
By treating document genre as a primary lens, NLP pipelines avoid the pitfalls of local context ambiguity. It transforms the messy, unorganized web into a structured repository where downstream models can process text with both high precision and optimized computational efficiency.
Key Foundational Work
BioGrapher: Biography Questions as a Restricted Domain Question Answering Task (2004) — contributed 2 concept elements to this direction.
Autonomously semantifying wikipedia (2007) — contributed 1 concept elements to this direction.