Introduction
Extracting structured event information from unstructured text is a cornerstone of advanced natural language processing. This paper details Texas A&M University's (TAMU) approach to the TAC KBP 2017 Event Nugget Detection and Coreference Resolution track. The researchers focus on "event nuggets"—the specific words or phrases (triggers) that signify an event, such as "merged" or "marry."
The task requires solving four integrated subproblems:
- Span Detection: Identifying the trigger's character span.
- Type Classification: Categorizing the event (e.g., Attack, Transfer-money).
- Realis Status Classification: Determining if the event is Actual, Generic, or Other (hypothetical).
- Coreference Resolution: Linking mentions of the same event across a document.
Methodology
The TAMU system employs a pipeline of three neural network-based classifiers. Unlike many contemporary systems that rely on complex deep learning architectures like RNNs or CNNs, this approach emphasizes the use of distinct syntactic features derived from dependency parse trees.
- Joint Span and Realis Classifier: An ensemble of 10 feed-forward neural networks that simultaneously predicts event spans and their realis status.
- Event Subtype Classifier: An ensemble of 3 networks categorizing events into 19 classes.
- Coreference Resolution: A pairwise classifier using shared neural layers to embed event attributes and calculate similarity for greedy merging.
The system's strength lies in its feature engineering, specifically leveraging the dependency relations and Part-of-Speech (POS) tags of a trigger's modifiers and governors.
Key Insights: Syntactic Signatures
A primary finding of this work is that event triggers possess unique "syntactic signatures." The distribution of dependency relations for event triggers differs markedly from non-events. For example, triggers are frequently associated with relations like ccomp (clausal complement) or nsubjpass (passive nominal subject).
Furthermore, the researchers found that:
- Named Entities Predict Types: The types of named entities modifying a trigger (e.g., Location for Attack events) are strong predictors of the event subtype.
- Joint Modeling Benefits: Predicting spans and realis status together improves precision, as temporal modifiers (e.g., "Wednesday") often signal both the event's existence and its status.
- Ensemble Robustness: Using ensembles of diverse neural networks helped mitigate overfitting and smoothed out performance variance.
Results and Evaluation
The system was evaluated using the TAC KBP 2015-2017 datasets, which include both structured news and informal discussion forum posts.
- KBP 2016 Benchmark: The joint span+realis model achieved a 53.47 F1 score, outperforming isolated classifiers. The feature-based ensemble reached 56.14 F1 on span detection, surpassing complex RNN/CNN-based state-of-the-art systems by approximately 1.5%.
- Official KBP 2017 Results: The system's Run I achieved top scores in span detection (57.72 F1) and realis classification (42.47 F1). Run II led in coreference resolution with a CoNLL F1 of 27.20.
Qualitatively, the system performed better on well-structured news than on noisy forum posts, where lack of punctuation often led to sentence segmentation and dependency parsing errors.
Conclusion
The TAMU system demonstrates that well-engineered syntactic features and joint modeling can outperform more complex neural architectures in event extraction tasks. Future work will focus on making these dependency-based features more robust to the noise found in informal text and further exploring joint modeling across all extraction subtasks.
Summary updated