Incremental Event Detection via Knowledge Consolidation Networks

Pengfei Cao, Yubo Chen, Jun Zhao, Taifeng Wang

Abstract

Event Detection (ED) is a critical task in Information Extraction, traditionally performed in a static setting. However, real-world applications require models to learn new event types incrementally. This paper introduces the Knowledge Consolidation Network (KCN), a novel framework designed to mitigate catastrophic forgetting in incremental ED. KCN employs Prototype Enhanced Retrospection (PER) to select representative examples for memory and Hierarchical Distillation (HD) to maintain knowledge across feature and prediction levels. Experimental results on ACE 2005 and TAC KBP 2017 benchmarks demonstrate that KCN significantly outperforms existing state-of-the-art incremental learning methods.

Introduction

Event Detection (ED) involves identifying event triggers in text and classifying them into predefined categories. While deep learning models have achieved high performance in static ED, they struggle with the dynamic nature of real-world data where new event types emerge over time. Simply fine-tuning a model on new data leads to "catastrophic forgetting," where the model loses its ability to recognize previously learned event types.

Existing replay-based methods for incremental learning often fail in the NLP domain due to semantic ambiguity and class imbalance. Words can have different meanings depending on context, and the influx of new data can easily overwhelm the limited memory of old examples. To address these challenges, we propose the Knowledge Consolidation Network (KCN).

Methodology: The Knowledge Consolidation Network

KCN is built upon a BERT backbone and incorporates two key modules to ensure stable incremental learning.

Prototype Enhanced Retrospection (PER)

To combat semantic ambiguity, KCN uses a prototype-based selection strategy for its memory buffer. Instead of random sampling, the model calculates a "prototype" vector—the mean representation of all instances in a class. It then selects the examples closest to this prototype as the most representative "exemplars." As new classes are added, the memory budget is dynamically reallocated by pruning the least prototypical examples of older classes.

Hierarchical Distillation (HD)

To address class imbalance and prevent weight drift, KCN employs a two-tier distillation process:

  1. Feature-level Distillation: This ensures that the latent representations generated by the BERT encoder for old data remain consistent with the previous model version.
  2. Prediction-level Distillation: This aligns the output logits of the current model with those of the previous model, preserving the inter-class relationships learned earlier.
Architecture of the Knowledge Consolidation Network (KCN) featuring Prototype Enhanced Retrospection and Hierarchical Distillation.

Experimental Setup

The performance of KCN was evaluated on two adapted benchmarks:

  • ACE 2005: The top 10 most frequent event types were introduced sequentially.
  • TAC KBP 2017: Similarly, the top 10 frequent classes were used for a class-incremental stream.

KCN was compared against several baselines, including Elastic Weight Consolidation (EWC), Learning without Forgetting (LwF), and Experience Management and Replay (EMR).

Results and Discussion

KCN demonstrated superior performance across both benchmarks. On the ACE dataset, KCN achieved a Whole F1 score of 55.1%, representing a 19% improvement over the EMR baseline. On the TAC KBP dataset, it reached a 34.9% Whole F1 score, outperforming EMR by 13.4%.

Performance comparison of KCN against various baselines on the ACE and TAC KBP benchmarks as the number of classes increases.

Ablation studies confirmed the importance of both PER and HD. Replacing prototype selection with random selection led to a significant performance drop, highlighting that the quality of remembered examples is more important than quantity in NLP tasks. Furthermore, the hierarchical distillation proved essential for stabilizing the BERT-based architecture during incremental updates.

Conclusion

The Knowledge Consolidation Network (KCN) effectively addresses the challenges of incremental event detection by combining representative memory selection with multi-level knowledge distillation. By focusing on prototypical examples, KCN avoids the pitfalls of semantic ambiguity inherent in language. Future work will explore extending this framework to more complex structural tasks such as Event Argument Extraction and investigating the impact of class ordering in lifelong learning scenarios.

Summary updated

BibTeX
@inproceedings{cao2020incremental,
  author = {Pengfei Cao and Yubo Chen and Jun Zhao and Taifeng Wang},
  title = {{Incremental Event Detection via Knowledge Consolidation Networks}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.52},
}
Figure