Abstract
Identifying specific domain-heavy segments within a document is a critical yet challenging task in NLP. While document-level classification is common, pinpointing the exact sentences or phrases that justify a label—fine-grained domain detection—suffers from an annotation bottleneck. This paper introduces DETNET, a hierarchical neural framework designed to detect domain-specific sentences and words using only "weak" document-level supervision. By integrating domain definitions as prior knowledge and employing Multiple Instance Learning (MIL), DETNET achieves robust performance across languages and genres, facilitating applications such as domain-aware summarization.
Introduction
Standard text classification typically treats documents as monolithic entities. However, real-world documents often comprise a mixture of general language and domain-specific information. For instance, a legal news report may blend common narrative with specialized terminology. Labeling every sentence in large-scale datasets is prohibitively expensive. To address this, we propose the task of fine-grained domain detection using weak supervision, where the model learns to identify "needles" of domain evidence in a "haystack" of general text.
Methodology: The DETNET Framework
The proposed model, DETNET (Domain Detection Network), utilizes an encoder-detector architecture designed to capture context at multiple granularities.
Encoder-Detector Architecture
DETNET consists of a hierarchical encoder and a corresponding detector. The encoder, based on the Transformer architecture, processes text at the word, sentence, and document levels. The detector then uses these representations to assign domain scores.
Multiple Instance Learning and Scoring
The core of DETNET's weak supervision is Multiple Instance Learning (MIL). The model treats the document as a "bag" and its constituent sentences and words as "instances." It learns to attribute the document-level label to specific instances by calculating hierarchical scores.
A key innovation is the integration of prior knowledge. The Word Detector incorporates domain definitions (short descriptive texts) to guide the identification of domain-heavy words, even in unfamiliar contexts. A gated mechanism balances signals from the raw input text against these external definitions, enhancing the model's generalization capabilities.
Experimental Setup and Datasets
The model was primarily trained on Wikipedia datasets in both English and Chinese, covering seven domains: Business, Government, Health, Law, Lifestyle, Military, and General Purpose. To evaluate sentence-level precision without gold-standard labels, the authors created a "Silver Standard" synthetic dataset by mixing domain-heavy lead sentences. Generalization was further tested on the New York Times (NYT) corpus, and human evaluations were conducted via Amazon Mechanical Turk to validate word and sentence-level detections.
Key Results
- Sentence-Level Performance: DETNET achieved a 54.37% Macro-F1 score on English Wikipedia sentences, significantly outperforming standard MIL baselines (37.12%) and hierarchical models without word-level signals (30.01%).
- Cross-Genre Generalization: The model maintained a strong 60.62% F1 score on the NYT dataset, demonstrating that the inclusion of domain definitions prevents overfitting to training-specific vocabulary.
- Qualitative Word Retrieval: DETNET successfully identified highly specific domain terms (e.g., "biosecurity") compared to broader topic modeling approaches like Labeled LDA.
- Application in Summarization: The authors introduced DETRANK, a domain-aware summarization tool. Human evaluators preferred DETRANK's summaries for their informativeness and coherence compared to standard graph-based methods.
Conclusion
DETNET demonstrates that hierarchical modeling combined with weak supervision and prior knowledge is highly effective for fine-grained domain detection. Future work may explore semi-supervised learning to further refine performance and address label ambiguity in multi-domain contexts.
Summary updated