REAR: Reinforced Reasoning Optimization for Event Argument Extraction with Relation-Aware Support

Jianwen Luo, Yu Hong, Shuai Yang, Jianmin YaoEMNLP 2025

TL;DR

This paper introduces REAR, a novel framework that enhances Event Argument Extraction (EAE) by leveraging Relation Extraction (RE) signals. REAR uses a two-stage approach: joint supervised fine-tuning to warm up a Large Language Model (LLM) with reasoning data, followed by reinforcement learning to optimize the integration of RE cues for better EAE. This approach significantly improves EAE performance by capturing inter-task semantic relationships.

What Problem Does This Paper Address?

Event Argument Extraction (EAE) aims to identify and classify arguments within an event context (e.g., who did what to whom, where, and when). Relation Extraction (RE) focuses on classifying the semantic relationships between entities. Traditionally, these tasks are handled independently, leading to a missed opportunity: RE can provide valuable contextual clues that help disambiguate event roles in EAE. For instance, knowing that two entities have a "physical" relationship might inform whether one is a "victim" or "place" in an event. Existing methods struggle to effectively integrate these complementary signals, often resulting in incomplete understanding and suboptimal EAE performance.

Methods

REAR employs a two-stage optimization strategy:

  1. Joint Supervised Warm-up: This initial stage uses a combined dataset of EAE and RE examples, augmented with automatically generated reasoning explanations. A single LLM policy is trained on this data using supervised fine-tuning (SFT). This "warm-up" phase helps the LLM learn to perform both tasks and incorporate auxiliary RE information.
  2. Joint Reinforced Optimization: Building on the warm-up, this stage uses Reinforcement Learning (RL), specifically Proximal Policy Optimization (PPO). The LLM dynamically explores diverse reasoning trajectories, iteratively refining its decision-making process. This RL approach allows the model to discover near-optimal strategies for integrating relation-level signals into EAE, overcoming the limitations of static SFT. Sparse, terminal-state rewards are defined for both EAE and RE to guide the learning process.

Datasets

The paper evaluates REAR on three standard datasets:

  • ACE-E: A subset of the ACE2005 corpus.
  • ACE-E+: An extended version of the ACE2005 corpus.
  • ERE: Event Relation Extraction dataset.

These datasets are commonly used for sentence-level evaluation of EAE and RE tasks.

Key Insights and Ideas

  • Synergy between EAE and RE: The core idea is that RE provides crucial semantic context that can significantly improve EAE. By jointly optimizing these tasks, REAR captures these inter-task dependencies.
  • Reinforcement Learning for Complex Integration: Standard supervised methods struggle to fully exploit the nuanced relationships between EAE and RE. RL, with its ability to explore diverse reasoning paths and learn from trial-and-error, is a more effective paradigm for this complex integration.
  • Reasoning-Enhanced Data: Augmenting training data with explicit reasoning steps (generated by LLMs) helps the model learn how to connect information, not just what the answer is.
  • Reward Shaping is Crucial: The design of reward functions in RL is critical. REAR uses carefully crafted rewards for both EAE and RE, including penalties for verbosity and sparse rewards for complete correctness, to guide the LLM towards accurate and concise outputs.

Key Results

REAR consistently outperforms previous decoder-only LLM methods on the ACE-E, ACE-E+, and ERE benchmarks. Specifically, it achieves F1-score gains of at least 0.9%, 2.2%, and 1.6% respectively. The two-stage approach shows significant improvements: the supervised warm-up alone yields substantial gains, and the subsequent RL optimization further refines performance, demonstrating the effectiveness of both stages. Ablation studies show that mixed-task training is superior to single-task training, and the RL stage further boosts performance.

Limitations and Future Work

The authors acknowledge several limitations:

  • Hyperparameter Tuning: REAR requires careful tuning of hyperparameters, which might limit its generalizability to new tasks.
  • Computational Overhead: RL-based optimization introduces higher computational costs compared to traditional supervised methods.
  • Sentence-Level Focus: The current framework primarily considers pairwise relationships within individual sentences, neglecting more complex cross-sentence or document-level semantic interactions.

Future work includes exploring alternative model architectures (e.g., encoder-decoder, encoder-only), investigating RL methods that require less annotated reasoning data, and extending the approach to handle document-level EAE.

Summary updated

BibTeX
@inproceedings{luo2025rear,
  author = {Jianwen Luo and Yu Hong and Shuai Yang and Jianmin Yao},
  title = {{REAR: Reinforced Reasoning Optimization for Event Argument Extraction with Relation-Aware Support}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure