Word-Label Alignment for Event Detection: A New Perspective via Optimal Transport

Amir Pouran Ben Veyseh, Thien NguyenNAACL 2022

TL;DR

This paper reframes Event Detection (ED) as a word-label alignment problem, moving beyond traditional sequence labeling. By leveraging Optimal Transport (OT) and introducing novel "word preference" and "type preference" biases, the authors achieve state-of-the-art performance on the ACE 2005 dataset, demonstrating the power of global alignment and bias incorporation.

What Problem Does This Paper Address?

Event Detection (ED) is a crucial task in Information Extraction, aiming to identify mentions of real-world events in text and classify them with specific event types (e.g., "Birth," "Attack"). Traditionally, ED has been approached as a sequence labeling problem, where each word in a sentence is assigned a label (like "B-Birth," "I-Birth," or "Other" for non-event words). While effective, these methods often treat word-label assignments locally, focusing on individual word predictions.

This paper argues that existing ED models are suboptimal because they:

  1. Lack global alignment: They don't fully leverage the global structure of word-label relationships across an entire sentence.
  2. Miss assignment biases: They fail to incorporate intuitive biases, such as certain words being more likely to be event triggers or certain event types appearing more frequently in a given context.

The authors propose a new perspective: modeling ED as a word-label alignment task, where the goal is to infer an "alignment matrix" that captures the likelihood of each word being associated with each possible event type. This matrix can then be used to derive global training signals and incorporate these missing biases.

Methods

The core of the proposed method, named OTED (Optimal Transport for Event Detection), lies in its novel formulation and the use of Optimal Transport (OT).

  1. Word and Label Representation:
    • Words: Pre-trained BERT embeddings are used to represent words in a sentence.
  2. Labels: Event type labels are represented by randomly initialized embeddings that are learned during training.
  3. Alignment Matrix Computation:
    • Vanilla Alignment: Previous ED models implicitly compute alignment scores (often through dot products of word and label embeddings) but don't explicitly form a global alignment matrix.
  4. Novel Biases: OTED introduces two key biases:
  5. Word Preference: Words that are more likely to be event triggers should have higher alignment scores with event types (and lower scores with the "Other" label). This is quantified using a Trigger Identification (TI) task, where a feed-forward network predicts trigger likelihood for each word.
  6. Type Preference: Event types that are more likely to appear in a sentence should have higher alignment scores with words. This is quantified using a Type Prediction (TP) task, which predicts the likelihood of each event type appearing in the sentence.
  7. Optimal Transport (OT): OT is employed to elegantly combine word/label representations and the two preference biases into a single alignment matrix. Specifically, the word preference scores form a distribution over words, and type preference scores form a distribution over event types. OT then finds the optimal "transport plan" (the alignment matrix) that minimizes the cost of transforming one distribution into the other, using Euclidean distance between word and type embeddings as the cost function. The Sinkhorn algorithm is used for approximate OT solution.
  8. Training Signals:
    • Task Loss ($L_{task}$): A standard cross-entropy loss is used, treating the OT-derived alignment score for a word and its gold label as a probability.
  9. OT Loss ($L_{OT}$): To enforce global alignment, the Wasserstein distance between the predicted alignment matrix and a "golden" binary alignment matrix (derived from gold labels) is minimized. This encourages the model's learned alignment to closely resemble the true alignment.
  10. Bias Losses ($L_{TI}$, $L_{TP}$): Binary cross-entropy losses are used to supervise the trigger likelihood (Word Preference) and type appearance likelihood (Type Preference) predictions.

The overall loss is a weighted sum of these components.

Datasets

The paper evaluates its method on the ACE 2005 dataset. This dataset is a standard benchmark for event detection, containing annotated documents with 33 event types. The authors use the same data split and preprocessing as prior work for fair comparison. No new datasets are introduced.

Key Insights and Ideas

  • Reframing ED as Alignment: The most significant insight is the shift from sequence labeling to a word-label alignment perspective. This opens up new avenues for incorporating global information and biases.
  • Optimal Transport for Bias Integration: Using OT is a clever way to fuse diverse information sources (word embeddings, type embeddings, and learned preferences) into a coherent alignment matrix. It provides a principled way to balance different objectives.
  • The Power of Biases: Both "Word Preference" and "Type Preference" biases are shown to be crucial. Word preference helps distinguish triggers from non-triggers, while type preference helps focus on relevant event types within a sentence.
  • Global Alignment Matters: The $L_{OT}$ loss, which directly minimizes the difference between predicted and golden alignment matrices, significantly contributes to performance, highlighting the benefit of global alignment signals.
  • Beyond Local Predictions: By moving beyond local cross-entropy losses for each word, the model can learn richer representations and relationships.

Key Results

  • State-of-the-Art Performance: OTED achieves state-of-the-art performance on the ACE 2005 dataset, outperforming strong baselines like BiLSTM, DMBERT, BERT+CRF, and ED3C. The paper reports a precision of 79.28%, recall of 79.48%, and F1 of 79.38% for the full OTED model.
  • Ablation Studies Confirm Contributions:
    • Removing either Word Preference or Type Preference significantly hurts performance, demonstrating their individual importance.
  • Excluding any of the loss components ($L_{task}$, $L_{OT}$, $L_{TI}$, $L_{TP}$) also leads to performance degradation, indicating that all components are beneficial.
  • Variants that directly minimize Wasserstein distance or use a simpler alignment distance metric perform worse than the proposed $L_{OT}$, underscoring the effectiveness of the chosen OT loss formulation.
  • Qualitative Improvements: Case studies show OTED successfully identifying event types and triggers where baselines fail, particularly by avoiding unlikely event types (due to type preference) and correctly identifying triggers while ignoring non-triggers (due to word preference).

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the current OT formulation uses Euclidean distance as the cost function, which might not capture all nuances of word-label relationships.
  • While OTED outperforms ED3C (which uses additional document context), the paper doesn't explicitly compare OTED's performance with models that do leverage document-level context.
  • Future Work:
    • The authors plan to evaluate their method on more datasets to better understand its generalizability.
  • They also aim to explore alternative cost functions for OT and investigate how to incorporate document-level context more effectively within their alignment framework.
  • Further exploration of different alignment biases is also a potential direction.

Summary updated

BibTeX
@inproceedings{veyseh2022word,
  author = {Amir Pouran Ben Veyseh and Thien Nguyen},
  title = {{Word-Label Alignment for Event Detection: A New Perspective via Optimal Transport}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure