Abstract
Event factuality identification (EFI) is a critical task in natural language processing that involves determining the factual status of events mentioned in text. This paper introduces EF-AC-GAN, a novel framework that combines factor extraction with a Generative Adversarial Network (GAN) featuring auxiliary classification. By generating synthetic syntactic paths, the model addresses the inherent data imbalance in EFI datasets and improves performance on rare and complex event types.
Introduction
Determining whether an event is a certain fact, a possibility, or a negation is complex due to two primary challenges:
- Data Imbalance: Datasets like FactBank are dominated by "certain positive" (CT+) instances, making it difficult for models to learn rare categories like "possible" or "negative."
- Syntactic Complexity: Factuality often depends on "embedded events," where the status of one event is modified by another predicate (e.g., "doubts that...").
Methodology: The EF-AC-GAN Framework
The proposed framework operates in two stages: basic factor extraction and classification via an Auxiliary Classifier GAN (AC-GAN).
1. Factor Extraction
The system first identifies core components from the raw text: the event, Source Introducing Predicates (SIPs), sources, and cues (e.g., "not," "maybe"). It then extracts Shortest Dependency Paths (SDPs) that connect these elements, providing a grammatical skeleton for the model.
2. EF-AC-GAN Architecture
The core of the model is an AC-GAN designed to "hallucinate" synthetic syntactic paths for rare event types. This allows the model to learn the structural patterns of speculative or negative sentences even when real-world examples are scarce.
- The Generator ($G$): Creates synthetic SDPs conditioned on specific class labels, effectively augmenting the training data for minority classes.
- The Discriminator ($D$): A BiLSTM-based component that distinguishes between real and synthetic paths while simultaneously performing auxiliary classification tasks to determine event modality and polarity.
Experimental Results
Evaluated on the FactBank benchmark, EF-AC-GAN significantly outperformed traditional baselines like SVMs and standard BiLSTMs.
- Overall Performance: The model achieved a Macro-F1 score of 54.06, compared to 43.97 for SVM and 51.63 for a standard BiLSTM.
- Rare Class Improvement: For "Possible Positive" (PS+) events, the F1 score reached 46.17, nearly doubling the performance of traditional SVM methods (25.90).
- Embedded Events: The model showed superior handling of complex nested predicates, achieving an F1 of 43.56 on embedded sources.
Conclusion
EF-AC-GAN demonstrates that using GANs to generate synthetic syntactic structures is an effective strategy for overcoming data imbalance in EFI. While errors still occur—primarily due to initial source identification failures (accounting for 72.64% of errors)—the model's ability to learn the "shape" of speculative and negative statements marks a significant advancement. Future work will focus on refining the initial factor extraction process to further enhance performance.
Summary updated