Abstract
Word Sense Disambiguation (WSD) has historically been limited by the extreme granularity of standard sense inventories like WordNet, which often leads to low human agreement and data sparsity. This paper introduces the Coarse Sense Inventory (CSI), a mapping of WordNet synsets to 45 domain-based labels. By simplifying the sense space, CSI allows neural models to surpass the traditional 80% accuracy "ceiling," achieving a record 85.9% F1 score while significantly improving performance on rare and unseen words.
The Granularity Problem in WSD
The primary goal of Word Sense Disambiguation is to identify the specific meaning of a word in context—for instance, distinguishing whether "bank" refers to a financial institution or a riverbank. For decades, WordNet has served as the gold standard inventory. However, its high level of detail often becomes a hindrance. WordNet frequently distinguishes between senses that are nearly indistinguishable to human annotators, such as "street" as a thoroughfare versus "street" as the paved portion between sidewalks.
This excessive detail results in two critical bottlenecks:
- Low Inter-Annotator Agreement (IAA): Even experts struggle to agree on specific sub-senses, introducing noise into training data.
- Data Sparsity: Many fine-grained senses appear rarely in corpora like SemCor, making it difficult for machine learning models to generalize effectively.
Methodology: Constructing CSI
The authors developed CSI through a systematic two-step mapping process designed to balance breadth and specificity.
1. Tagset Definition
Rather than creating new labels from scratch, the researchers leveraged Roget’s Thesaurus, which contains 1,075 categories. These were clustered by linguists into 45 coarse labels (e.g., LAW&CRIME, OLFACTORY, ART&ARCHITECTURE). Unlike traditional taxonomic hierarchies based on "is-a" relationships (hypernymy), CSI labels are domain-based, focusing on the topical context of a word.
2. Synset Mapping
WordNet synsets were then mapped to these 45 labels. The process involved manual mapping of the most frequent synsets in the SemCor corpus, followed by an expansion using existing resources like BabelDomains to cover 83,000 WordNet synsets across all parts of speech.
Evaluation and Key Results
To evaluate CSI, the researchers trained standard neural WSD architectures, including Bi-LSTMs and BERT-based models, to predict coarse labels instead of fine-grained senses.
Breaking the Performance Ceiling
A BERT-based model utilizing CSI achieved an 85.9% F1 score across standard benchmarks (Senseval and SemEval). This represents a significant leap over the sub-80% scores typically seen in fine-grained WSD tasks.
Human Reliability and Descriptiveness
One of the most striking findings was the improvement in data quality. As shown in the table below, inter-annotator agreement (IAA) rose to 0.81 for CSI, compared to just 0.51 for WordNet. Furthermore, CSI labels were rated as more "descriptive" and intuitive than competing coarse inventories like WordNet Domains (WND).
Key Insights
- Domain over Hierarchy: Focusing on "what a word is about" (e.g., a "poodle" belongs to
ANIMALS) rather than its strict taxonomic lineage proves more effective for broad disambiguation. - Solving the "Long Tail": Because CSI labels are shared across many words, models can "borrow" semantic knowledge. A model that learns the
BUSINESS&ECONOMICSlabel for "bank" can apply that understanding to rare terms like "arbitrage" seen during inference. - Few-Shot Learning: CSI significantly outperformed other inventories in scenarios with limited training examples (3-5 instances per word), demonstrating superior generalization.
Limitations and Future Outlook
While CSI offers a powerful tool for broad semantic understanding, it involves an inherent trade-off. By grouping senses, it loses the ability to distinguish between closely related but distinct meanings (e.g., "sidewalk" vs. "roadway"). If an application requires extreme precision, CSI may be too coarse.
Looking forward, the authors suggest that CSI’s domain-based nature makes it an ideal candidate for cross-lingual WSD, as high-level concepts like FOOD&DRINK are likely to remain consistent across languages even when specific lexical usages vary.
Summary updated