Supersense Tagging of Unknown Nouns in WordNet

Massimiliano Ciaramita, Mark JohnsonEMNLP 2003 · OpenReview

Abstract

We present a new framework for classifying common nouns that extends named-entity classification. We used a fixed set of 26 semantic labels, which we called supersenses. These are the labels used by lexicographers developing WordNet. This framework has a number of practical advantages. We show how information contained in the dictionary can be used as additional training data that improves accuracy in learning new nouns. We also define a more realistic evaluation procedure than cross-validation.

Introduction: Beyond Named Entities

While Named Entity Recognition (NER) is a staple of NLP, it typically focuses on proper nouns like people, organizations, and locations. However, common nouns (e.g., "solvent", "iris", "artifact") often lack semantic labels in broad-coverage systems, despite their importance for tasks like question answering or coreference resolution.

Full WordNet sense disambiguation is notoriously difficult due to the sheer volume of specific synsets (over 66,000). This paper proposes a "Goldilocks" level of abstraction: supersense tagging. Instead of pinpointing a specific leaf node, the system identifies which of the 26 broad "lexicographer classes" a word belongs to.

The 26 lexicographer classes used as supersense labels in WordNet.

Methodology: The Averaged Perceptron

The authors employ a Multiclass Averaged Perceptron classifier. This online learning algorithm is "ultraconservative," updating weights only upon error. The "averaging" step mitigates overfitting by using the mean of all weight vectors generated during the training process.

The feature set incorporates:

  • Contextual clues: Surrounding words, n-grams, and part-of-speech tags.
  • Syntactic relations: Governing verbs or objects.
  • Morphology: Prefixes, suffixes, and complex word structures.

Leveraging Dictionary Data

A key innovation is the use of WordNet's own internal data. The authors treat WordNet's glosses (definitions) and example sentences as additional training instances. The rationale is that the language used to define an artifact (like "chair") provides a high-signal, low-noise template for identifying other artifacts in natural text.

Evaluation and Results

To ensure a realistic assessment, the authors avoided standard cross-validation. Instead, they used a "temporal" split: training on WordNet 1.6 (the "old" state) and testing on 744 unambiguous nouns added in WordNet 1.71. This mimics the real-world challenge of an evolving lexicon.

Performance comparison of different models on the temporal test sets.

As shown in the results, the baseline "most frequent class" approach performs poorly (21-28%). The proposed model, particularly when augmented with WordNet internal data (AP-B-55+WN), achieves a significant boost, reaching approximately 53% accuracy on noun types.

Conclusion

Supersense tagging provides a practical middle ground for semantic classification. By utilizing dictionary definitions as a source of "free" training data, the system achieves robust performance on unknown nouns. Future work will focus on extending this framework to polysemous words and refining features to better distinguish between abstract categories like communication and cognition.

Summary updated

Abstract

We present a new framework for classifying common nouns that extends named-entity classification. We used a fixed set of 26 semantic labels, which we called supersenses. These are the labels used by lexicographers developing WordNet. This framework has a number of practical advantages. We show how information contained in the dictionary can be used as additional training data that improves accuracy in learning new nouns. We also define a more realistic evaluation procedure than cross-validation.
BibTeX
@inproceedings{ciaramita2003supersense,
  author = {Massimiliano Ciaramita and Mark Johnson},
  title = {{Supersense Tagging of Unknown Nouns in WordNet}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2003},
  url = {https://openreview.net/forum?id=BJbcQGz_WB},
}
Figure