Learning Cross-Task Dependencies for Joint Extraction of Entities, Events, Event Arguments, and Relations

Minh Van Nguyen, Bonan Min, Franck Dernoncourt, Thien Huu Nguyen

Abstract

Information Extraction (IE) typically involves identifying entities, events, relations, and event arguments. While joint models aim to solve these tasks simultaneously to avoid error propagation, they often rely on rigid, hand-crafted heuristics to link different information types. This paper introduces DepIE (Dependency-guided JointIE), a framework that automatically learns complex cross-task dependencies from data. By leveraging multi-layer language model representations and the Chow-Liu algorithm, DepIE discovers statistical patterns between information types and instances. Experimental results across multiple languages and datasets demonstrate that DepIE significantly outperforms existing state-of-the-art joint extraction models.

Introduction

The goal of Information Extraction is to transform unstructured text into structured knowledge. This involves four sub-tasks: entity extraction, event detection, relation extraction, and event argument extraction. Traditional pipeline approaches suffer from "cascading errors," where a mistake in an early stage (like entity detection) ruins subsequent steps. Joint models address this by performing all tasks at once. However, current joint models often use human-designed rules to define how tasks interact. These rules are frequently suboptimal and fail to generalize across different languages.

Methodology: Learning Dependencies

DepIE replaces manual heuristics with a learned dependency structure. The architecture consists of three primary stages:

  1. Instance Detection: The model uses a Pretrained Language Model (PLM) such as BERT, followed by Conditional Random Fields (CRFs), to identify candidate spans for entities and events.
  2. Cross-Instance Dependencies: Instead of a fixed graph, DepIE treats every instance as a node in a fully connected graph. Edge weights are learned by aggregating representations from multiple layers of the PLM, capturing both syntactic and semantic relationships. A Graph Convolutional Network (GCN) then refines these node representations.
  3. Cross-Type Dependencies: The model employs the Chow-Liu algorithm to learn a global dependency tree of information types (e.g., the likelihood of a "Victim" argument appearing with a "Die" event).
The DepIE architecture for joint information extraction.

To bridge local sentence instances with global type patterns, DepIE uses random walk graph kernels. This allows the model to measure the similarity between the current sentence's dependency graph and the "ideal" patterns learned from the entire training set.

Experimental Setup

The model was evaluated on several multilingual benchmarks:

  • ACE 2005: English (E+), Chinese (CN), and Arabic (AR).
  • ERE: Spanish (ES).
  • BETTER: English (EN) and Farsi (FA).

These datasets provide a rigorous test for the model's ability to handle diverse event types and linguistic structures.

Key Results

DepIE consistently sets new state-of-the-art performance across all tasks and languages.

Performance comparison of DepIE against state-of-the-art models across multiple languages.

As shown in the results table, DepIE achieves significant gains over strong baselines like OneIE and FourIE. Notably:

  • In English (ACE05-E+), it improved Argument Extraction F1 to 61.2 and Relation Extraction to 64.9.
  • The improvements are even more dramatic in non-English settings. For instance, in Spanish (ERE-ES), Argument Extraction F1 improved by 6.8 points over FourIE.
  • The model also demonstrated superior performance in cross-lingual transfer (training on English and testing on other languages), suggesting that the learned dependencies capture universal information structures.

Discussion and Insights

The success of DepIE highlights two major insights:

  • PLM Layers as Evidence: Internal transformer layers contain rich information about how words relate. By weighting dependencies based on these layers, the model avoids the need for external dependency parsers or manual rules.
  • Statistical Anchors: The use of the Chow-Liu algorithm allows the model to discover "anchor knowledge"—global statistical patterns of how events and entities interact—which provides a robust prior for classification.

Conclusion

DepIE demonstrates that learning cross-task dependencies directly from data is more effective than relying on manual heuristics. By combining multi-layer PLM features with statistical type-level patterns, the model achieves state-of-the-art performance across multiple languages and IE sub-tasks. Future work may involve extending this framework to higher-order dependencies and additional tasks like coreference resolution.

Summary updated

BibTeX
@inproceedings{nguyen2022learning,
  author = {Minh Van Nguyen and Bonan Min and Franck Dernoncourt and Thien Huu Nguyen},
  title = {{Learning Cross-Task Dependencies for Joint Extraction of Entities, Events, Event Arguments, and Relations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.634},
}
Figure