OEE-CFC: A Dataset for Open Event Extraction from Chinese Financial Commentary

Qizhi Wan, Changxuan Wan, Rong Hu, Dexi Liu, Xu Wenwu, Kang Xu, Zou Meihua, Liu Tao, Jie Yang, Zhenwei XiongEMNLP 2024 · DOI

Introduction

Event Extraction (EE) is a critical task in natural language processing, traditionally focused on identifying simple entities (who, what, where) within predefined event types. However, real-world financial commentary presents a much higher level of complexity. A single sentence in a financial report might describe an event’s background, the specific manner of its occurrence, the intensity of the action, and the resulting state change.

Existing datasets often struggle with three primary limitations: simplistic ontologies that miss critical context (like "manner" or "background"), simple arguments that fail to account for complex phrases or nested events, and fixed schemas that cannot handle the diverse nature of financial discourse. To address these gaps, the OEE-CFC dataset treats event extraction as a deep semantic understanding task rather than a pattern-matching exercise.

Methodology: The 21-Role Ontology

The researchers developed a general open event template inspired by semantic role labeling. This template utilizes a rich ontology of 21 argument roles to capture the intricate logic of financial text. These roles are categorized into three groups:

  • Core Roles: Subject and Object.
  • Situational Roles: Time, Location, Background, Manner, Scope, Material, Tool, Source, Result, Reason, Purpose, and State.
  • Adverbial Roles: Degree, Scope, Time, Place, Affirmative, Negative, and Modal adverbs.
Distribution of the 21 argument roles within the OEE-CFC dataset.

To evaluate the dataset, the authors employed state-of-the-art extraction frameworks, including PAIE (a prompt-based model) and TabEAE (a table-generation model). They found that "Explicit Continuous" prompts—which describe roles in natural language—provided the best semantic hints for the models to achieve high performance.

Dataset Characteristics

The OEE-CFC dataset is sourced from financial commentary articles published by the "Forum of 50 Economists." It comprises 4,253 documents containing 17,469 events and 44,221 arguments.

A defining feature of OEE-CFC is its logical density. Unlike the widely used ChFinAnn dataset, where approximately 71% of documents contain only a single event, OEE-CFC reflects the complexity of professional financial analysis: over 62% of its documents contain four or more events.

Comparison of event density per document between ChFinAnn and OEE-CFC.

Key Insights and Semantic Complexity

The dataset highlights several phenomena unique to high-level financial discourse:

  1. The Necessity of Adverbs: Omitting adverbs like "possibly" (modal) or "already" (time) can completely distort the meaning of a financial event—for instance, distinguishing between a predicted market shift and one that has already occurred.
  2. Event Nesting: In financial text, events are often interconnected. One event (e.g., "grain prices rising") can serve as the subject or cause of another event (e.g., "driving up the CPI"). OEE-CFC is specifically designed to capture these nested structures.
  3. Semantic-Driven Extraction: By moving beyond simple entity extraction to a 21-role ontology, the authors argue that EE can move closer to full sentence comprehension, preserving semantic clarity that is otherwise lost.

Results and Performance

The dataset maintains high reliability, with an average Inter-Annotator Agreement (Kappa score) of 84.88% across all roles and 91.2% for event triggers. In model evaluations, the PAIE model using the Explicit Continuous prompt achieved the top results:

  • Head F1 (Identification/Classification): 75.92 / 69.51
  • Span F1 (Identification/Classification): 66.35 / 63.47

Conclusion and Future Work

OEE-CFC represents a significant advancement in Open Event Extraction for the financial domain. While currently limited to Chinese financial commentary, it provides a robust benchmark for capturing complex arguments and nested logic. Future work may involve expanding the corpus to other languages and domains, as well as explicitly annotating the causal and temporal relationships between extracted events.

Summary updated

BibTeX
@inproceedings{wan2024oee,
  author = {Qizhi Wan and Changxuan Wan and Rong Hu and Dexi Liu and Xu Wenwu and Kang Xu and Zou Meihua and Liu Tao and Jie Yang and Zhenwei Xiong},
  title = {{OEE-CFC: A Dataset for Open Event Extraction from Chinese Financial Commentary}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.256},
}
Figure