Lifelong Event Detection with Knowledge Transfer

Pengfei Yu, Heng Ji, Prem NatarajanEMNLP 2021 · OpenReview

Abstract: Pengfei Yu, Heng Ji, Prem Natarajan. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.

Traditional event detection systems are often limited by static ontologies, making them difficult to adapt as new event categories emerge. This paper introduces a Lifelong Event Detection (LED) framework designed to continuously learn new event types while mitigating catastrophic forgetting and addressing the long-tail problem of rare events. By implementing a two-way knowledge transfer mechanism, the model uses existing knowledge to "warm-start" new types and leverages new data to refine old knowledge. The approach achieves a 5.1% overall F1 boost and a 30% gain for rare events.

The Challenge of Evolving Ontologies

In Information Extraction (IE), models are typically trained on fixed sets of event types. However, real-world applications require systems to adapt to new categories (e.g., adding "Pandemic" to a health-related model). Simply retraining from scratch is inefficient, while incremental updates usually lead to:

  1. Catastrophic Forgetting: The model loses its ability to detect previously learned types.
  2. The Long-Tail Problem: New types often lack sufficient training data, making them difficult to learn from scratch.

Methodology: Two-Way Knowledge Transfer

The proposed framework treats event detection as a continuous process, building upon standard techniques like Experience Replay and Knowledge Distillation. The core innovation is a Two-Way Knowledge Transfer module that exploits the semantic relatedness between event types.

Lifelong Event Detection framework illustrating two-way knowledge transfer.

1. Old to New (Knowledge-aware Initialization)

Instead of random initialization, the model uses the representations of semantically related old types to initialize new ones. For instance, a new "Charge" type can be initialized using the "Trial" type's representation, providing a "semantic anchor" that is crucial when data is scarce.

2. New to Old (Self-training)

The model uses new data to update old knowledge. When a mention of a new type is encountered, the model checks its relevance to previous categories using a self-training loss with "soft" labels, ensuring that the understanding of old types remains current.

3. Gating Mechanism

A mathematical gate determines the balance between relying on old knowledge versus new data. For frequent types, the model relies more on data; for rare "long-tail" types, it leans more heavily on knowledge transfer from existing categories.

Evaluation and Results

The framework was tested on the ACE 2005 and MAVEN datasets, partitioned into five stages to simulate a lifelong learning environment.

Macro F1 performance across training stages for old, new, and rare types.

The results demonstrate that the framework significantly outperforms competitive baselines:

  • Overall Performance: A 5.1% absolute F1 improvement on the MAVEN dataset.
  • Long-Tail Success: Rare event types (fewer than 120 mentions) saw an F1 boost of over 30%.
  • Stability: As shown in the performance plots, the model (KD+R+K) maintains high F1 scores for "Old" types while rapidly improving on "New Rare" types compared to standard distillation methods (KD+R).
  • Robustness: The model performed well even in "Silver Negative" settings, simulating realistic scenarios where new training data is not perfectly annotated for old types.

Conclusion

By leveraging the inherent correlations within event ontologies, this framework moves beyond simple preservation of knowledge to active improvement. While challenges like domain shift remain, the success in boosting performance for rare events marks a significant step toward more robust and adaptable IE systems.

Summary updated

BibTeX
@inproceedings{yu2021lifelong,
  author = {Pengfei Yu and Heng Ji and Prem Natarajan},
  title = {{Lifelong Event Detection with Knowledge Transfer}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=r8vEGf7f9Ix},
}
Figure