Event Factuality Identification via Generative Adversarial Networks with Auxiliary Classification

Zhong Qian, Peifeng Li, Yue Zhang, Guodong Zhou, Qiaoming ZhuIJCAI 2018 · OpenReview

Abstract

Event factuality identification (EFI) is a critical task in natural language processing that involves determining the factual status of events mentioned in text. This paper introduces EF-AC-GAN, a novel framework that combines factor extraction with a Generative Adversarial Network (GAN) featuring auxiliary classification. By generating synthetic syntactic paths, the model addresses the inherent data imbalance in EFI datasets and improves performance on rare and complex event types.

Introduction

Determining whether an event is a certain fact, a possibility, or a negation is complex due to two primary challenges:

  1. Data Imbalance: Datasets like FactBank are dominated by "certain positive" (CT+) instances, making it difficult for models to learn rare categories like "possible" or "negative."
  2. Syntactic Complexity: Factuality often depends on "embedded events," where the status of one event is modified by another predicate (e.g., "doubts that...").

Methodology: The EF-AC-GAN Framework

The proposed framework operates in two stages: basic factor extraction and classification via an Auxiliary Classifier GAN (AC-GAN).

1. Factor Extraction

The system first identifies core components from the raw text: the event, Source Introducing Predicates (SIPs), sources, and cues (e.g., "not," "maybe"). It then extracts Shortest Dependency Paths (SDPs) that connect these elements, providing a grammatical skeleton for the model.

Example of a dependency tree and the resulting extracted paths for an embedded event.

2. EF-AC-GAN Architecture

The core of the model is an AC-GAN designed to "hallucinate" synthetic syntactic paths for rare event types. This allows the model to learn the structural patterns of speculative or negative sentences even when real-world examples are scarce.

  • The Generator ($G$): Creates synthetic SDPs conditioned on specific class labels, effectively augmenting the training data for minority classes.
  • The Discriminator ($D$): A BiLSTM-based component that distinguishes between real and synthetic paths while simultaneously performing auxiliary classification tasks to determine event modality and polarity.
The EF-AC-GAN architecture, illustrating how the Generator and Discriminator process syntactic paths and class labels.

Experimental Results

Evaluated on the FactBank benchmark, EF-AC-GAN significantly outperformed traditional baselines like SVMs and standard BiLSTMs.

  • Overall Performance: The model achieved a Macro-F1 score of 54.06, compared to 43.97 for SVM and 51.63 for a standard BiLSTM.
  • Rare Class Improvement: For "Possible Positive" (PS+) events, the F1 score reached 46.17, nearly doubling the performance of traditional SVM methods (25.90).
  • Embedded Events: The model showed superior handling of complex nested predicates, achieving an F1 of 43.56 on embedded sources.

Conclusion

EF-AC-GAN demonstrates that using GANs to generate synthetic syntactic structures is an effective strategy for overcoming data imbalance in EFI. While errors still occur—primarily due to initial source identification failures (accounting for 72.64% of errors)—the model's ability to learn the "shape" of speculative and negative statements marks a significant advancement. Future work will focus on refining the initial factor extraction process to further enhance performance.

Summary updated

Abstract

Event factuality identification is an important semantic task in NLP. Traditional research heavily relies on annotated texts. This paper proposes a two-step framework, first extracting essential factors related with event factuality from raw texts as the input, and then identifying the factuality of events via a Generative Adversarial Network with Auxiliary Classification (AC-GAN). The use of AC-GAN allows the model to learn more syntactic information and address the imbalance among factuality values. Experimental results on FactBank show that our method significantly outperforms several state-of-the-art baselines, particularly on events with embedded sources, speculative and negative factuality values.
BibTeX
@inproceedings{qian2018event,
  author = {Zhong Qian and Peifeng Li and Yue Zhang and Guodong Zhou and Qiaoming Zhu},
  title = {{Event Factuality Identification via Generative Adversarial Networks with Auxiliary Classification}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2018},
  url = {https://openreview.net/forum?id=S14XsBz_Wr},
}
Figure