Reinforcement Learning-Based Dialogue Guided Event Extraction to Exploit Argument Relations

Qian Li, Hao Peng, Jianxin Li, Jia Wu, Yuanxing Ning, Lihong Wang, Philip S. Yu, Zheng Wang

TL;DR

This paper introduces a novel framework that treats event extraction as a multi-turn, task-oriented dialogue. By using reinforcement learning to optimize the order in which arguments are extracted, the model explicitly exploits correlations between event participants, significantly outperforming state-of-the-art methods on the ACE 2005 benchmark.

What Problem Does This Paper Address?

Event Extraction (EE) is the task of identifying "triggers" (the occurrence of an event) and their "arguments" (the participants, locations, or attributes involved). For example, in the sentence "The man detonated explosives in the car," the trigger is "detonated," and the arguments include "man" (attacker), "explosives" (instrument), and "car" (target).

The core challenge is that an argument's role often depends heavily on context and its relationship with other arguments. Most existing methods extract arguments either simultaneously or in a fixed sequence, largely ignoring these inter-argument correlations. In isolation, a word like "troops" could be an attacker or a target; identifying a "weapon" elsewhere in the sentence provides the "aha!" moment needed to settle that ambiguity. The authors aim to bridge this gap by modeling these dependencies through an iterative, dialogue-driven process.

Methods

The authors propose a framework consisting of three main components:

  1. Double Perspective Event Representation: To understand the sentence, the model combines two views:
  2. Lexicon-based Graph Attention Network (GAT): Captures global semantics and long-range dependencies by treating words as nodes in a graph.
  3. Event-based BERT: Learns contextual representations of words specifically tailored for event extraction.
  4. Ranked Argument Extraction (The Dialogue System): The task is formulated as a conversation between two agents:
  5. Agent A (The Questioner): Generates queries based on the event schema (e.g., "What is the killing instrument?").
  6. Agent B (The Answerer): Identifies the span in the text that answers the query.
  7. Reinforcement Learning (RL): An RL policy determines the optimal order of questions. By extracting "easier" or more certain arguments first, the system builds a context that makes extracting "harder" arguments more accurate.
  8. Incremental Event Learning: As arguments are extracted, they are fed back into the model as "pseudo-labels" and "pseudo-edges" in the GAT. This updates the sentence representation in real-time, allowing the model to "understand" the sentence better with every turn of the dialogue.

Datasets

The model was evaluated on the ACE 2005 dataset, the standard benchmark for event extraction. It contains 599 documents annotated with 8 coarse-grained event types, 33 subtypes, and 36 argument roles. The authors tested their approach across English, Arabic, and Chinese portions of the dataset.

Key Insights and Ideas

  • Order Matters: One of the paper's most significant insights is that the sequence of extraction is a hidden variable that impacts performance. Using RL to navigate the "search space" of argument roles allows the model to leverage the most informative arguments first.
  • Dialogue as a Natural Interface: Formulating EE as a dialogue makes it easier to incorporate domain knowledge (via question templates) and maintain a "history" of what has already been discovered in a sentence.
  • Dynamic Representation: Unlike static models, this framework's understanding of a sentence evolves. The "Incremental Learning" aspect means the model isn't just finding pieces of a puzzle; it's re-evaluating the whole picture as each piece is placed.

Key Results

The "Full Model" consistently outperformed seven state-of-the-art methods, including JMEE and MQAEE.

  • Accuracy Gains: On the Argument Role Classification (ARC) task, the model achieved an F1-score of 61.42%, a significant jump over previous machine reading comprehension-based models (~53-59%).
  • Ablation Success: Removing the Reinforcement Learning (RLD) module led to the most drastic performance drop, confirming that the optimized extraction order is the "secret sauce" of this framework.
  • Robustness: The model showed particular strength in scenarios with limited labeled data, thanks to the information-rich nature of the dialogue-guided approach.

Limitations and Future Work

  • Computational Complexity: Because the model relies on multiple rounds of dialogue and RL-based optimization, it has a higher training and inference latency compared to "one-shot" extraction models. The authors noted a runtime of 38.1 hours for their full model compared to ~20-24 hours for simpler baselines.
  • Scattered Arguments: The model occasionally struggles when arguments are distributed far apart in complex sentences containing multiple events. In these cases, the "historical context" of the dialogue can sometimes lose track of distant relations.
  • Future Directions: The authors suggest incorporating commonsense knowledge into the dialogue system to further resolve ambiguities that aren't explicitly stated in the text. They also aim to optimize the model's efficiency to reduce training difficulty.

Summary updated

Abstract

Event extraction is a ftask for natural language processing. Finding the roles of event arguments like event participants is essential for event extraction. However, doing so for real-life event descriptions is challenging because an argument’s role often varies in different contexts. While the relationship and interactions between multiple arguments are useful for settling the argument roles, such information is largely ignored by existing approaches. This paper presents a better approach for event extraction by explicitly utilizing the relationships of event arguments. We achieve this through a carefully designed task-oriented dialogue system. To model the argument relation, we employ reinforcement learning and incremental learning to extract multiple arguments via a multi-turned, iterative process. Our approach leverages knowledge of the already extracted arguments of the same sentence to determine the role of arguments that would be difficult to decide individually. It then uses the newly obtained information to improve the decisions of previously extracted arguments. This two-way feedback process allows us to exploit the argument relations to effectively settle argument roles, leading to better sentence understanding and event extraction. Experimental results show that our approach consistently outperforms seven state-of-the-art event extraction methods for the classification of events and argument role and argument identification.
BibTeX
@article{li2022reinforcement,
  author = {Qian Li and Hao Peng and Jianxin Li and Jia Wu and Yuanxing Ning and Lihong Wang and Philip S. Yu and Zheng Wang},
  title = {{Reinforcement Learning-Based Dialogue Guided Event Extraction to Exploit Argument Relations}},
  journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
  year = {2022},
  doi = {10.1109/taslp.2021.3138670},
}
Figure