TL;DR
Word Sense Disambiguation (WSD) has long been split between scalable, unannotated knowledge-based approaches and high-performing, data-hungry corpus-based classifiers. This paper demonstrates that rather than treating them as competitors, combining a WordNet-based Specification Marks (SM) method with a supervised Maximum Entropy (ME) classifier significantly improves disambiguation accuracy, achieving near state-of-the-art results across English and Spanish benchmarks.
What Problem Does This Paper Address?
In Natural Language Processing, Word Sense Disambiguation (WSD) is the task of determining which sense of a polysemous word is intended in a specific context (e.g., whether plant refers to a factory or a living organism).
Historically, WSD has relied on two isolated paradigms:
- Knowledge-based methods: Leverage structured lexical resources like WordNet. They require no expensive sense-annotated training data, but their precision typically lags far behind supervised methods.
- Corpus-based (supervised) methods: Train statistical or machine learning models on human-annotated examples. While they achieve higher precision, they suffer from the knowledge acquisition bottleneck—annotating thousands of words across dozens of senses and multiple languages is prohibitively expensive.
The authors explore whether these two approaches can play complementary roles, testing whether lexical knowledge can enrich statistical classifiers and vice versa.
Methods
The paper evaluates two primary algorithms and several ways to integrate them:
- Specification Marks (SM) Method (Knowledge-Based): Disambiguates nouns using WordNet’s IS-A hierarchy. It identifies the most specific common concept subsuming the candidate senses of words in a context window. To boost recall and resolve ties, SM incorporates five heuristics: hypernym/hyponym depth weighting, gloss overlap, gloss-hypernym overlap, common specification marks, and domain-based vector similarity (via WordNet Domains).
- Maximum Entropy (ME) Framework (Corpus-Based): Formulates WSD as a conditional classification problem. It extracts local context features (neighboring words, POS tags, collocations, grammatical dependencies, and keywords). The authors introduce collapsed features, which merge linguistic patterns across occurrences to reduce feature sparsity.
- Integration Strategies:
- Corpus $\to$ Knowledge: A high-precision ME model pre-tags unambiguous context nouns ($\ge 90\%$ accuracy) before SM runs, drastically narrowing SM’s hierarchical search space.
- Knowledge $\to$ Corpus: SM’s domain heuristic assigns coarse semantic domain labels to context words, which are fed into ME as domain features.
- Ensemble Voting (
vME+SM): Multiple ME classifiers (trained with different feature-selection scopes) are combined with the SM method in a majority-voting ensemble.
Datasets
The authors evaluated their approaches on standard multilingual benchmarks:
- SemCor: A subset of the Brown Corpus annotated with WordNet senses, used to evaluate SM heuristics and train ME context classifiers.
- SENSEVAL-2 (Spanish Lexical-Sample Task): 17 nouns, verbs, and adjectives used to benchmark feature configurations and system combinations.
- SENSEVAL-2 (English Lexical-Sample Task): 29 target nouns evaluated under the integration and ensemble schemes.
- Auxiliary Resources: WordNet 2.0 and WordNet Domains (165 hierarchical domain labels).
Key Insights and Ideas
- High Complementarity Despite Low Agreement: Error analysis showed that the ME and SM models rarely agreed on difficult instances (Cohen’s $\kappa \le 0.14$). However, the theoretical ceiling of an optimal combination was remarkably high (up to $89\%$). In fact, SM successfully resolved 15% of cases where ME failed completely, proving that hierarchical semantic networks capture cues statistical models miss.
- Feature Selection per Part-of-Speech Beats Word-Level Selection: When tuning ME features, selecting features per part-of-speech (
MEbfs.pos) generalized better than selecting features per word (MEbfs). Per-word feature optimization overfit due to limited training samples. - Domain Context Is a Double-Edged Sword: Adding domain features produced double-digit accuracy gains for words with distinct domain boundaries (e.g., art, authority, spade). However, for broad, domain-neutral terms (such as stress, tied largely to the generic
FACTOTUMdomain), domain features added noise and reduced accuracy by up to $6.5\%$.
Key Results
- Ensemble Performance (
vME+SM): On the Senseval-2 Spanish task, adding SM to the ME ensemble boosted noun accuracy to 0.702, tying the top-ranked system at Senseval-2 (Johns Hopkins University) and ranking 2nd overall (accuracy 0.684). - Corpus-Assisted SM: Pre-tagging context words with ME increased SM’s precision on English nouns from 30.0% to 33.6% and recall from 26.7% to 30.3%.
- Domain-Assisted ME: Adding domain features to ME on the English lexical sample produced a statistically significant overall precision boost from 64.9% to 66.9% without requiring extensive feature tuning.
- Heuristic Voting: Combining all five knowledge-based heuristics via majority voting lifted SM recall on polysemous SemCor nouns to 39.1%, outperforming standard Lesk ($27.4\%$) and Conceptual Density ($22.0\%$).
Limitations and Future Work
- Noun Restriction in Knowledge Methods: The Specification Marks method relies entirely on WordNet’s noun taxonomy, leaving verbs and adjectives dependent solely on the corpus-based classifier.
- Weak Verb Performance: Both ME and combined systems struggled significantly on verbs (accuracy hovered between $51\%$ and $58\%$), indicating that local collocations and bag-of-words contexts are insufficient for complex verbal semantics.
- Domain Sensitivities: Because domain features degraded accuracy on 7 of the 29 tested English nouns, the authors suggest implementing per-word dynamic gating to apply domain filtering only when senses are cleanly partitioned by topic.
Summary updated