A Corpus and Model Integrating Multiword Expressions and Supersenses

Nathan Schneider, Noah A. SmithNAACL 2015 · OpenReview

TL;DR

This paper tackles the challenge of understanding word meaning by jointly identifying multiword expressions (MWEs) and assigning them semantic supersense labels. They introduce a new dataset and a sequence tagging model that achieves a respectable 70% F1 score for class labeling, demonstrating the value of integrating these two crucial aspects of lexical semantics.

What Problem Does This Paper Address?

Computational lexical semantics aims to go beyond simply recognizing words to understanding their meanings in context. While resources like WordNet offer hierarchical semantic categories (supersenses), annotating text at the token level with these categories is labor-intensive and often limited by WordNet's coverage. Furthermore, many meaningful units in language are not single words but multiword expressions (MWEs) like idioms or phrasal verbs. These MWEs, due to their holistic nature, should ideally be treated as single units for semantic analysis. Previous work often treated MWE identification and supersense tagging separately, or relied on automatically mapping WordNet senses to supersenses, which can be problematic. This paper addresses the gap of jointly identifying MWEs and assigning them supersenses in a unified framework, particularly in noisy, real-world text like online reviews.

Methods

The core of this paper is a joint sequence tagging model that simultaneously predicts both MWE boundaries and their supersense labels. They leverage a first-order structured perceptron with averaging, a standard discriminative modeling approach. The model uses a Viterbi search to find the most likely tag sequence for a sentence.

What's novel here is the integrated tagset. Instead of separate tags for MWEs and supersenses, they use an 8-way positional flag scheme (extending the BIO scheme) to encode MWE status (single-word, start of MWE, continuation of strong/weak MWE, and handling of gappy MWEs). These positional flags are then combined with supersense class labels. This allows the model to learn dependencies between MWE structure and semantic meaning. They also introduce new features that consult WordNet supersenses of related words and check for the presence of supersenses, aiming to improve performance, especially for ambiguous cases.

Datasets

The primary dataset used is the STREUSLE dataset, which is an extension of the REVIEWS corpus. This corpus, approximately 55,000 words, was already annotated for multiword expressions (from the CMWE corpus) and has now been further annotated with noun and verb supersenses. This represents a new benchmark for joint MWE and supersense annotation. They also mention using the SemCor dataset for an out-of-domain baseline comparison.

Key Insights and Ideas

  • Joint modeling is beneficial: The paper argues that MWEs and supersenses are intrinsically linked and should be modeled together. Their results show that jointly modeling them leads to only a minor decrease in MWE identification performance compared to modeling them separately, and this decrease is largely recovered with richer features.
  • Supersenses are a practical abstraction: Supersenses provide a coarse-grained, broadly applicable semantic label space that complements MWEs, offering a more scalable approach to lexical semantics than fine-grained sense disambiguation.
  • Handling gappy MWEs is crucial: Existing supersense taggers often struggle with discontinuous expressions (e.g., "track people down"). Their 8-way tagset explicitly accounts for these "gappy" MWEs, improving coverage.
  • Annotation quality matters: The paper emphasizes the importance of detailed annotation guidelines and rigorous quality control (double annotation and reconciliation) for building reliable datasets, especially for verbs where new guidelines were developed.

Key Results

The main quantitative result is achieving 70% class labeling F1 on the STREUSLE test set with their best model. This is a strong baseline for the joint task, especially considering the relatively small, noisy training corpus.

  • The best model, incorporating new supersense-aware features, achieved an F1 score of 82.49% for overall tagging accuracy.
  • They demonstrate that their joint model performs significantly better than an out-of-domain baseline trained on SemCor (achieving ~50% F1 vs. ~65% F1 for their in-domain model).
  • Qualitatively, their system can handle gappy expressions and distinguishes between similar-sounding but semantically different lemmas (e.g., "look" for V:COGNITION vs. V:PERCEPTION).

Limitations and Future Work

  • Domain Specificity: The current work is focused on the "online reviews" genre. The authors acknowledge that building robust lexical semantic models that work across many domains will require further experimentation and more diverse annotated resources.
  • Feature Engineering: While they introduced new features, there's always room for improvement. Future work could explore richer features, potentially incorporating syntactic information (though they deliberately avoided it for modularity).
  • Model Complexity: They imposed a percept cutoff to manage model size, suggesting that exploring more efficient model architectures or learning algorithms could be beneficial.
  • Indirect Supervision: The authors suggest that future investigations could explore indirect forms of supervision, which might alleviate the need for extensive manual annotation.
  • Downstream Tasks: They anticipate that their integrated approach will be valuable for downstream tasks like semantic parsing and machine translation.

Summary updated

Abstract

This paper introduces a task of identifying and semantically classifying lexical expressions in running text. We investigate the online reviews genre, adding semantic supersense annotations to a 55,000 word English corpus that was previously annotated for multiword expressions. The noun and verb supersenses apply to full lexical expressions, whether single- or multiword. We then present a sequence tagging model that jointly infers lexical expressions and their supersenses. Results show that even with our relatively small training corpus in a noisy domain, the joint task can be performed to attain 70% class labeling F1.
BibTeX
@inproceedings{schneider2015corpus,
  author = {Nathan Schneider and Noah A. Smith},
  title = {{A Corpus and Model Integrating Multiword Expressions and Supersenses}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
  url = {https://openreview.net/forum?id=H1VXzQ-_WH},
}
Figure