Introduction
In Natural Language Processing (NLP), understanding word meaning is a core challenge. Historically, researchers have developed various tasks to measure this understanding, most notably Word Sense Disambiguation (WSD), Word-in-Context (WiC), and Target Sense Verification (TSV). While these tasks are clearly related, the community has long debated whether they measure fundamentally different aspects of semantics or are merely variations of the same underlying problem.
In their NAACL 2022 paper, Bradley Hauer and Grzegorz Kondrak provide a formal proof and empirical evidence that these three tasks are theoretically equivalent. By establishing the "sense-meaning hypothesis," they demonstrate that these tasks are essentially different ways of asking the same question, suggesting that models and datasets can be shared across these previously distinct sub-fields.
The Three Tasks of Lexical Semantics
To understand the equivalence, we must first define the three tasks:
- Word Sense Disambiguation (WSD): Given a word $w$ in a context $C$, the goal is to select the correct sense $s$ from a predefined dictionary (e.g., WordNet).
- Word-in-Context (WiC): Given two contexts $C_1$ and $C_2$ containing the same word $w$, the goal is to determine if $w$ has the same meaning in both instances (a binary True/False decision).
- Target Sense Verification (TSV): Given a word $w$ in context $C$ and a specific sense $s$, the goal is to decide if $s$ is the correct sense for $w$ in $C$.
Formal Equivalence via Problem Reduction
The authors utilize the concept of problem reduction from theoretical computer science. If an algorithm for Problem B can be used to solve Problem A, then A is reducible to B. They propose the sense-meaning hypothesis: two instances of a word have the same meaning if and only if they share the same sense in a formal inventory.
As shown in the figure above, the reductions work as follows:
- WiC to WSD: To determine if a word has the same meaning in two sentences, run a WSD algorithm on both. If the resulting sense IDs match, the answer is "True."
- WSD to TSV: To find the correct sense, perform a TSV check against every possible sense in the dictionary. The sense that returns "True" is the correct one.
- TSV to WiC: To verify a sense, compare the target word against a "gold standard" example sentence for that sense using a WiC algorithm.
Empirical Validation
The authors tested these reductions using state-of-the-art systems: LIORI for WiC and ESCHER for WSD. They utilized standard benchmarks including MCL-WiC, SemEval 2007, and WordNet 3.0.
The results strongly supported the theoretical framework:
- Solving WiC with WSD: A WSD model achieved 78.5% accuracy on the MCL-WiC task, nearly matching the theoretical prediction of 77.9%.
- Solving WSD with WiC: Using a WiC-based reduction on SemEval 2007 yielded 47.9% accuracy—significantly higher than the 34% expected by chance, confirming the practical viability of the reduction.
- Manual Audit: A manual review of 60 instances showed that 91.7% of cases perfectly aligned with the equivalence hypothesis.
Insights and Limitations
The study reveals that human judgments of "same meaning" align remarkably well with the fine-grained distinctions in dictionaries like WordNet. However, the primary bottleneck remains the quality of the sense inventory itself.
As illustrated in the table above, different dictionaries (OED, MW, Wik) provide varying levels of granularity. When reductions fail, it is often due to missing senses (e.g., a specific technical usage not present in WordNet) or annotation errors in the datasets, rather than a failure of the underlying theory.
Conclusion
By proving the equivalence of WSD, WiC, and TSV, Hauer and Kondrak provide a unifying framework for lexical semantics. This finding encourages the NLP community to move toward standardized sense inventories and multi-task learning approaches, where a single model can effectively address all three semantic challenges.
Summary updated