ESC: Redesigning WSD with Extractive Sense Comprehension

Edoardo Barba, Tommaso Pasini, Roberto NavigliNAACL 2021 · DOI

Abstract

Word Sense Disambiguation (WSD) has traditionally been framed as a classification task over fixed sense inventories, which often leads to issues with rare senses and inventory rigidity. This paper introduces Extractive Sense Comprehension (ESC), reframing WSD as a span extraction task where the model identifies the correct definition from a concatenated list of candidates. Our proposed model, ESCHER, achieves a new state-of-the-art F1 score of 80.7 on standard benchmarks and demonstrates superior performance in zero-shot and few-shot scenarios, outperforming existing models with significantly less training data.

Introduction

Word Sense Disambiguation (WSD) is the foundational NLP task of determining the specific meaning of a word based on its context. Despite its importance, traditional approaches treat WSD as a multi-label classification problem, which suffers from three primary limitations: the use of meaningless discrete labels, poor performance on the "long tail" of rare senses, and a dependency on rigid dictionary inventories (like WordNet) that require complete retraining to change.

Methodology: Extractive Sense Comprehension

To overcome these limitations, the authors propose the Extractive Sense Comprehension (ESC) framework. Inspired by extractive Question Answering (QA), the model is provided with a context sentence and a concatenated string of all possible definitions (glosses) for the target word. The task is then transformed into identifying the start and end indices of the most appropriate definition.

The ESCHER architecture reframes WSD as a span extraction task over candidate definitions.

The ESCHER Model

The implementation, named ESCHER, utilizes a BART-large transformer architecture. By processing the context and all candidate definitions simultaneously, the model's self-attention mechanism can perform "cross-definition" reasoning, directly comparing meanings against the target context. To mitigate "Most Frequent Sense" (MFS) bias, the authors introduced Gloss Noise (GN)—a training technique that injects random, frequent definitions as negative signals to force the model to rely on semantic understanding rather than frequency statistics.

Experimental Results

ESCHER was evaluated using the standard WSD evaluation suite, including SemCor for training and the "all-words" datasets (Senseval and SemEval) for testing.

Comparison of ESCHER against baselines and prior work on standard WSD benchmarks.

Performance and Efficiency

As shown in the results table, ESCHER achieved a new state-of-the-art F1 score of 80.7 on the "ALL" benchmark. Beyond raw performance, the model demonstrated remarkable data efficiency: in few-shot settings, ESCHER trained on a single example per sense outperformed the previous best model (BEM) trained on five examples.

Zero-Shot and Cross-Inventory Flexibility

Because ESCHER treats WSD as a reading task rather than a labeling task, it generalizes exceptionally well to unseen data. On the "0-def" dataset (definitions never seen during training), ESCHER outperformed the previous state-of-the-art by nearly 9 F1 points (76.8 vs 68.2). Furthermore, the model is inventory-agnostic; the authors successfully trained it on WordNet and Oxford Dictionary data simultaneously, reaching an F1 of 81.5 and proving that diverse definitions enhance contextual understanding.

Conclusion

By reframing WSD as extractive reading comprehension, ESCHER moves the field away from rigid classification heads toward flexible, gloss-aware reasoning. While limited by the context window of current transformers and a focus on sentence-level rather than document-level context, the ESC framework provides a robust path forward for more generalizable and data-efficient word sense disambiguation.

Summary updated

BibTeX
@inproceedings{barba2021esc,
  author = {Edoardo Barba and Tommaso Pasini and Roberto Navigli},
  title = {{ESC: Redesigning WSD with Extractive Sense Comprehension}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.371},
}
Figure