Event Causality Identification via Generation of Important Context Words

Hieu Man, Minh Nguyen, Thien NguyenNAACL 2022

TL;DR

This paper tackles Event Causality Identification (ECI) by reframing it as a generative task. Instead of just classifying if two events are causal, their model generates the causal label and the crucial words in the sentence that support this causal link, leading to state-of-the-art performance. This generative approach allows for multi-task learning, improving the model's understanding of context for better causal prediction.

What Problem Does This Paper Address?

Event Causality Identification (ECI) is a critical task in Information Extraction (IE) that aims to determine if one event causes another within a given text. For example, in "Massive fires cause major damages," an ECI system should identify that "fires" cause "damages." While previous deep learning models have achieved good performance, they primarily treat ECI as a classification problem. This means they output only a binary label (causal or not causal) and don't explicitly identify why they made that decision. The paper argues that this is a significant limitation because it misses the opportunity to leverage the important contextual words (like "cause" in the example) that are key to understanding the causal relationship. Current classification models don't explicitly output these crucial context words, hindering their ability to learn more robust representations.

Methods

The core innovation of this paper is framing ECI as a generative sequence-to-sequence problem. Instead of a classifier, they use a generative model (specifically, the T5 transformer) that takes the input sentence and the two event mentions and generates an output sequence. This output sequence combines both the causal label (e.g., "Yes") and the words along the dependency path between the two event mentions in the sentence's dependency tree. This dependency path is treated as the "important context words."

This approach enables multi-task learning: the model simultaneously learns to predict the causal label and generate the dependency path words. To train this generative model effectively, they employ the REINFORCE algorithm, a policy-gradient method. This is crucial because it allows them to directly optimize for the desired performance metric (causal prediction accuracy) rather than just maximizing the likelihood of the generated sequence. They design several novel reward functions for REINFORCE:

  • Performance-based Reward (Rper): Directly rewards correct causal label prediction.
  • Output-based Reward (Rout): Encourages similarity between the generated output and the golden output sequence (using ROUGE-2).
  • Input-based Reward (Rin): Promotes similarity between the generated output and the input sentence, leveraging the idea that important context words should be present in both.

Before REINFORCE training, they also perform maximum likelihood pre-training on the transformed input-output pairs. This helps to initialize the model and constrain the large action space of text generation, making the subsequent REINFORCE training more stable and effective.

Datasets

The paper evaluates their proposed model, GenECI, on two benchmark English datasets for ECI:

  • EventStoryLine: A corpus with 258 documents and 4316 sentences.
  • Causal-TimeBank: A corpus with 184 documents and 6813 event mentions.

They follow standard data splits for development and evaluation, using 5-fold cross-validation for EventStoryLine and 10-fold cross-validation for Causal-TimeBank.

Key Insights and Ideas

  • Reframing ECI as Generation is Powerful: Treating ECI as a generative task, rather than just classification, unlocks the ability to explicitly extract and learn from important contextual words. This is a significant departure from prior work.
  • Multi-task Learning with Dependency Paths Boosts Performance: Jointly generating the causal label and the dependency path words acts as a form of multi-task learning. The dependency path provides valuable contextual information that improves the primary task of causal prediction.
  • REINFORCE with Custom Rewards is Key: Directly optimizing for performance metrics using REINFORCE, with carefully designed rewards that balance causal accuracy and generation quality, is more effective than standard likelihood maximization for this generative ECI task.
  • Input-Output Similarity as a Reward: The input-based reward (Rin) is an interesting idea, suggesting that the model should learn to generate context that is semantically aligned with the input sentence, further reinforcing the importance of context.

Key Results

The proposed GenECI model achieves state-of-the-art performance on both benchmark datasets.

  • On EventStoryLine, GenECI achieved an F1 score of 58.8, outperforming the previous best model (RichGCN) by 3.6%.
  • On Causal-TimeBank, GenECI achieved an F1 score of 56.5, also outperforming previous models.

Ablation studies showed that all components of their model are important: the dependency path generation, the REINFORCE training with its specific reward functions, and the ML pre-training all contribute significantly to the overall performance. Removing the dependency path generation component led to a substantial drop in performance, verifying the effectiveness of their multi-task learning approach.

Limitations and Future Work

  • Limitations: The paper acknowledges that while their generative approach is effective, it's more complex than traditional classification methods. The REINFORCE training, while powerful, can be sensitive to hyperparameters and may require careful tuning. The paper also notes that ML training (without REINFORCE) can produce incorrect dependency paths, highlighting the challenges of pure likelihood maximization for this task.
  • Future Work: The authors plan to extend their generative model to other relation extraction tasks. This suggests a broader applicability of their generative framework beyond just event causality.

Summary updated

BibTeX
@inproceedings{man2022event,
  author = {Hieu Man and Minh Nguyen and Thien Nguyen},
  title = {{Event Causality Identification via Generation of Important Context Words}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure