From Incremental Meaning to Semantic Unit (phrase by phrase)

Andreas Scherbakov, Ekaterina Vylomova, Fei Liu, Timothy Baldwin

Abstract

This paper describes an experimental approach to Detection of Minimal Semantic Units and their Meaning (DiMSUM), explored within the framework of SemEval 2016 Task 10. The approach is primarily based on a combination of word embeddings and parser-based features, and employs unidirectional incremental computation of compositional embeddings for multiword expressions.

Introduction

In natural language processing, words are often treated as the fundamental "atoms" of meaning. However, the prevalence of Multiword Expressions (MWEs)—such as "kick the bucket" or "take a shower"—challenges this assumption, as their meanings are frequently non-compositional or idiomatic. The DiMSUM (Detection of Minimal Semantic Units and their Meaning) task addresses this by requiring systems to perform simultaneous segmentation (identifying MWE boundaries) and labeling (assigning broad semantic "supersenses" from WordNet, such as FOOD or MOTION).

Methodology

The proposed architecture utilizes a unidirectional incremental approach to build semantic representations. The system consists of three primary components:

  1. Incremental Vector Calculator: A specialized Recursive Neural Network (RNN) that constructs a vector representation ($v_n$) word-by-word. To maintain a consistent feature space, the model utilizes a "seed" vector even for single-word units.
  2. MWE Classifier: A two-layer perceptron (2LP) that evaluates the incremental vector to determine if the current sequence constitutes a valid MWE.
  3. Sense Classifier: A separate 2LP tasked with predicting the supersense of the identified semantic unit.
System architecture illustrating the incremental vector calculation and dual classification heads.

The model incorporates three distinct feature sets: pre-trained word2vec embeddings (Google News), syntactic parser features (e.g., hierarchical distance in a dependency tree from TurboParser), and character hashing to handle out-of-vocabulary (OOV) terms.

Experimental Setup and Results

The system was evaluated on the DiMSUM dataset, comprising approximately 4,800 POS-tagged English sentences. Official results indicated a high precision for MWE identification (61.22%), though recall remained lower (28.07%), leading to an MWE F1-score of 38.49%. Supersense labeling proved more robust, achieving an F1-score of 51.62%.

The Division of Labor: Syntax vs. Semantics

A key finding of this research is the distinct roles played by different feature types. Syntactic features, particularly distance-based metrics from the parser, are essential for MWE detection. The data suggests that MWEs rarely cross clause boundaries, making syntax a powerful boundary marker. In contrast, word embeddings are the primary driver for semantic sense prediction.

Ablation study results showing the impact of removing specific feature sets on MWE and Supersense performance.

As shown in the ablation study, removing word2vec embeddings causes a catastrophic drop in Supersense F1 (from ~52% to 18.29%), while having a less severe impact on MWE detection compared to the removal of distance or recurrency features.

Discussion and Conclusion

The incremental, greedy nature of the model introduces a "hysteresis" effect, where the system occasionally over-extends MWE boundaries. This suggests that future iterations would benefit from global context or lookahead mechanisms, such as bidirectional LSTMs or attention-based Transformers. Additionally, the "collision" between MWE detection and sense prediction within the same vector space suggests that these tasks might require more distinct representational paths. Despite these challenges, the study successfully demonstrates the complementary nature of syntactic structure and distributional semantics in identifying minimal semantic units.

Summary updated

Abstract

This paper describes an experimental approach to Detection of Minimal Semantic Units and their Meaning (DiMSUM), explored within the framework of SemEval 2016 Task 10. The approach is primarily based on a combination of word embeddings and parserbased features, and employs unidirectional incremental computation of compositional embeddings for multiword expressions.
BibTeX
@inproceedings{scherbakov2016from,
  author = {Andreas Scherbakov and Ekaterina Vylomova and Fei Liu and Timothy Baldwin},
  title = {{From Incremental Meaning to Semantic Unit (phrase by phrase)}},
  booktitle = {SemEval @ NAACL},
  year = {2016},
  url = {https://openreview.net/forum?id=FVmqJgcx58P},
}
Figure