Knowledge Enhanced Event Causality Identification with Mention Masking Generalizations

Jian Liu, Yubo Chen, Jun Zhao

Abstract

Event Causality Identification (ECI) is a critical task in natural language processing that involves determining causal relationships between events mentioned in text. Current approaches often suffer from "knowledge blindness"—ignoring external commonsense—and poor generalization due to overfitting on small datasets. This paper presents a dual-reasoning framework that integrates external knowledge from ConceptNet with a novel mention masking technique. By forcing the model to learn event-agnostic context patterns and enriching it with semantic relations, the proposed $M_{FULL}$ model achieves state-of-the-art performance and demonstrates significantly improved robustness when encountering unseen event predicates.

Introduction

Determining if one event causes another (e.g., an "earthquake" generating a "tsunami") is fundamental to understanding narrative structure. However, ECI faces two primary challenges. First, models typically rely only on local context, missing the broader commonsense knowledge humans naturally apply. Second, because ECI datasets are small, models often "memorize" specific event pairs rather than learning the linguistic indicators of causality. This leads to a performance collapse when the model is tested on events not seen during training.

Methodology: The Dual-Reasoning Framework

To address these limitations, the authors propose a model architecture consisting of two specialized reasoners balanced by an attentive gating mechanism.

The dual-reasoning architecture: $M_{KG}$ retrieves ConceptNet relations while $M_{MMR}$ processes masked sentences to learn context patterns.

Knowledge-Aware Reasoner ($M_{KG}$)

This module bridges the "knowledge blindness" gap by retrieving semantic paths from ConceptNet. For instance, if the text mentions an earthquake and a tsunami, $M_{KG}$ identifies relations like earthquake $\xrightarrow{Causes}$ tsunami. These paths are linearized into text and encoded via BERT to provide the model with context-independent background facts.

Mention Masking Reasoner ($M_{MMR}$)

To prevent overfitting on specific word pairs, the Mention Masking Reasoner replaces event words with a [MASK] token. This forces the model to focus on the "connective tissue" of the sentence—the event-agnostic patterns such as "The [MASK] generates a [MASK]". By employing Distributional Similarity Learning, the model ensures that different sentences expressing similar causal logic have consistent internal representations.

The Attentive Sentinel

The outputs of both reasoners are combined using a gating mechanism called the Attentive Sentinel. This component dynamically weights the contribution of each reasoner based on the input. If the specific events are ambiguous or missing from the knowledge base, the sentinel shifts more weight to the context-based patterns identified by $M_{MMR}$.

Experimental Results and Generalization

The model was evaluated on three standard benchmarks: EventStoryLine, Causal-TimeBank, and EventCausality. The full model ($M_{FULL}$) consistently outperformed previous state-of-the-art (SOTA) methods. On EventStoryLine, it achieved a 50.1% F1 score, marking a 5.5% improvement over the previous leading model, LIP.

F1 score comparison across different levels of event familiarity, showing the superior generalization of $M_{FULL}$ on unseen predicates.

The most striking result appeared in generalization tests. When tasked with identifying causality between events never seen during training ("Both Unseen"), the performance of previous SOTA models plummeted (e.g., LIP dropped to 11.3% F1). In contrast, $M_{FULL}$ maintained a much higher F1 score (55.6% in the "Both Unseen" category), proving that the combination of external knowledge and mention masking effectively teaches the model the underlying logic of causality rather than just word associations.

Conclusion

This research demonstrates that ECI can be significantly improved by balancing specific semantic knowledge with generalized linguistic patterns. By masking event mentions, the model is forced to become a "linguistic detective," identifying causal structures that remain robust even when the specific events change. Future work may involve expanding this dual-reasoning approach to other relational tasks, such as temporal extraction or general relation classification.

Summary updated

Abstract

Identifying causal relations of events is a crucial language understanding task. Despite many efforts for this task, existing methods lack the ability to adopt background knowledge, and they typically generalize poorly to new, previously unseen data. In this paper, we present a new method for event causality identification, aiming to address limitations of previous methods. On the one hand, our model can leverage external knowledge for reasoning, which can greatly enrich the representation of events; On the other hand, our model can mine event-agnostic, context-specific patterns, via a mechanism called event mention masking generalization, which can greatly enhance the ability of our model to handle new, previously unseen cases. In experiments, we evaluate our model on three benchmark datasets and show our model outperforms previous methods by a significant margin. Moreover, we perform 1) cross-topic adaptation, 2) exploiting unseen predicates, and 3) cross-task adaptation to evaluate the generalization ability of our model. Experimental results show that our model demonstrates a definite advantage over previous methods.
BibTeX
@inproceedings{liu2020knowledge,
  author = {Jian Liu and Yubo Chen and Jun Zhao},
  title = {{Knowledge Enhanced Event Causality Identification with Mention Masking Generalizations}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/499},
}
Figure