TL;DR
This paper introduces a novel annotation scheme and dataset for classifying event types in mythological narratives, addressing the challenge of their complex and non-linear structure. By manually annotating over 6,000 statements from diverse mythological sources into four categories (single-point, durative-constant, durative-initial, durative-resultative), they enable the training of a classifier to distinguish these event types, paving the way for computational analysis of myths.
What Problem Does This Paper Address?
Mythological narratives, unlike many modern stories, often present events in a non-straightforward manner. They employ stylistic devices like prolepses (foreshadowing) and allusions, making it difficult for computational methods to automatically extract and label events. Furthermore, the distinction between events (changes of state) and states (background knowledge or continuous conditions) is crucial for understanding narrative structure, but existing annotation schemes may not fully capture the nuances of mythological texts. This paper tackles the challenge of creating a robust annotation framework and dataset specifically for mythological narratives to enable their computational analysis.
Methods
The core of this work is the development of a "hylistic" approach to event annotation, building on the work of Zgoll (2019). This approach categorizes statements (called "hylemes") into four types:
- Single-point: Events that occur at a specific moment in time, with a clear beginning and end within the narrative sequence (e.g., actions, perceptions).
- Durative-constant: States that are true throughout a portion or the entirety of the narrative sequence, often conveying background knowledge (e.g., "Orpheus is the son of Oeagrus").
- Durative-initial: States that are true at the beginning of a sequence but may change later.
- Durative-resultative: States that become true at some point during the sequence and remain so thereafter, often as a result of an event.
The researchers manually extracted sequences of these hylemes from 228 mythological sources, representing plots and background information. These sequences were then annotated by six domain experts using the hylistic categories. A multinomial naive Bayes classifier was trained on this annotated data to automatically predict event types.
Datasets
The paper introduces a newly annotated dataset consisting of 6,315 German statements (hylemes) extracted from 228 mythological narrative sequences. These sequences originate from diverse geographical and cultural contexts, including Ancient Greece, Rome, and Mesopotamia. The dataset was manually curated and annotated by domain experts and serves as the basis for training and evaluating their event classifier.
Key Insights and Ideas
- Mythological narratives are inherently complex: The non-linear and allusive nature of myths poses significant challenges for standard event extraction techniques.
- Distinguishing events from states is crucial: The hylistic approach effectively differentiates between dynamic events and static background information, which is vital for narrative analysis.
- Durative states have subtle variations: The distinction between durative-constant, durative-initial, and durative-resultative states is context-dependent and requires careful annotation, highlighting the need for domain expertise.
- Manual annotation is essential for mythological texts: Due to the complexity and scarcity of some sources, manual extraction and annotation by domain experts are currently indispensable.
- Classifier performance is promising but has limitations: While a simple classifier can achieve reasonable performance, the inherent imbalance and context-dependency of some categories present ongoing challenges.
Key Results
- High Inter-Annotator Agreement: The annotation process achieved substantial to almost perfect inter-annotator agreement (Cohen's κ ranging from 0.61 to 0.99) for most annotator pairs, indicating the robustness of the annotation scheme.
- Classifier Performance:
- A binary classifier distinguishing single-point from durative events achieved an F1 score of 0.79 for single-point and 0.92 for durative events.
- A classifier for fine-grained durative types showed varying performance, with the highest F1 score for durative-constant (0.85) and lower scores for durative-initial (0.32) and durative-resultative (0.56).
- The fine-grained classifier generally favored the single-point class, misclassifying durative-constant statements as single-point in 70 cases.
- Data Distribution: The dataset is heavily skewed towards single-point statements, with durative-constant being the most frequent durative type.
Limitations and Future Work
- Manual Effort: The primary limitation is the significant manual effort required for extracting and annotating hyleme sequences, which limits scalability.
- Context Dependency: Durative labels, especially durative-initial and durative-resultative, are highly context-sensitive, making classification challenging.
- Data Imbalance: The skewed distribution of event types in the dataset impacts classifier performance.
- Future Work: The authors suggest exploring more sophisticated NLP models, such as multi-lingual transformers, to improve classification accuracy. They also envision using the annotated data for comparative analyses of mythological plots and background information, potentially including adaptations in other media like films. The ability to pre-classify new statements could also aid in future annotation efforts.
Summary updated