ClearTAC: Verb Tense, Aspect, and Form Classification Using Neural Nets

Skatje Myers, Martha PalmerACL 2019

TL;DR

This paper introduces ClearTAC, a neural network model for automatically classifying verb tense, aspect, and form in English. By outperforming a rule-based system on the PropBank dataset, ClearTAC offers a promising solution for enriching semantic representations like Abstract Meaning Representations (AMRs) with crucial temporal information, thereby streamlining annotation efforts and enabling better temporal reasoning.

What Problem Does This Paper Address?

Understanding the temporal nuances of verbs – their tense (e.g., present, past), aspect (e.g., progressive, perfect), and form (e.g., finite, participle) – is fundamental for accurately interpreting the sequence and structure of events in natural language. This information is vital for various downstream NLP applications, including question answering and narrative understanding. However, manually annotating this temporal information is labor-intensive. The authors aim to bridge this gap by developing an automated system that can accurately classify these verb properties, thereby facilitating the integration of temporal information into semantic representations like Abstract Meaning Representations (AMRs). AMRs, while powerful for capturing sentence meaning, have historically omitted tense and aspect, making temporal reasoning and generation from AMRs more challenging.

Methods

The core of the proposed solution is a Bidirectional Long Short-Term Memory (BiLSTM) with a Conditional Random Field (CRF) layer, named ClearTAC. This architecture is a standard choice for sequence labeling tasks in NLP. The BiLSTM processes the input sentence in both forward and backward directions, capturing contextual information from both sides of a word. The CRF layer then uses this contextualized representation to predict the most likely sequence of labels (form, tense, and aspect) for each verb token.

Key aspects of their approach include:

  • Joint Classification: Form, tense, and aspect are treated as a single, joint classification task, rather than being predicted independently.
  • Pre-trained Embeddings: The model utilizes pre-trained GloVe embeddings to represent each token, providing a rich semantic starting point.
  • No POS/Syntactic Pre-processing: The system operates directly on sentence-length sequences without relying on explicit part-of-speech tags or syntactic parses, simplifying the pipeline.

Datasets

The primary dataset used is PropBank I. This dataset provides annotations for semantic roles, roleset IDs, and verb inflections within the original Penn Treebank. The authors specifically leverage the annotations for verb form, tense, and aspect.

To address potential mismatches between their annotation goals and the TMV-annotator tool, they also created a Reduced Propbank I dataset. This reduced version excludes gerunds, ignores tense for participles with auxiliaries, and removes standalone participles without auxiliaries. This ensures a fairer comparison with the rule-based TMV-annotator.

Key Insights and Ideas

  • Neural Networks Excel at Temporal Nuances: The success of the BiLSTM-CRF model demonstrates that neural networks are highly effective at capturing the subtle distinctions in verb tense, aspect, and form, often outperforming traditional rule-based approaches.
  • Joint Classification is Effective: Treating form, tense, and aspect as a single, interconnected task rather than separate subtasks yields strong results, suggesting these features are highly interdependent.
  • Automated Annotation as a Shortcut: The paper strongly advocates for automated temporal annotation as a crucial pre-processing step to reduce the manual effort required for enriching semantic representations like AMRs. This can significantly speed up the annotation process for large corpora.
  • Rule-Based Systems Struggle with Nuance and Data Mismatches: The comparison with TMV-annotator highlights the limitations of rule-based systems, particularly when dealing with the full complexity of linguistic phenomena or when evaluated on data that doesn't perfectly align with their design.

Key Results

ClearTAC achieved impressive results on the PropBank I dataset:

  • Full Propbank I: The model achieved an F1 score of 93.46 for the combined task of classifying form, tense, and aspect.
  • Reduced Propbank I: The F1 score for the combined task was 95.63.
  • Outperforms Rule-Based System: ClearTAC significantly outperformed the rule-based TMV-annotator. On the reduced dataset (where TMV-annotator was designed to perform), ClearTAC achieved an F1 score of 95.63 compared to TMV-annotator's 73.92. On the full dataset, ClearTAC achieved 93.46 F1, while TMV-annotator scored 63.48.
  • High Performance on Subtasks: Individual subtask performance was also strong, with F1 scores in the mid-90s for form, tense, and aspect.

Limitations and Future Work

  • Out-of-Vocabulary (OOV) Words: The current model relies on pre-trained GloVe embeddings, which may struggle with OOV words. Future work suggests exploring embeddings like ELMo or BERT to improve performance on these cases and better distinguish between identical verb forms (e.g., gerunds vs. present-tense verbs).
  • Dataset Specificity: The current evaluation is primarily on PropBank I. The authors plan to evaluate their system on other datasets, such as TimeML, which is also annotated with tense and aspect information.
  • Task Decomposition: While joint classification was successful, the authors suggest that further improvements might be possible by dividing the subtasks of classifying form, tense, and aspect into separate models.
  • Architectural Improvements: Beyond embeddings, other architectural enhancements could be explored.

Summary updated

BibTeX
@inproceedings{myers2019cleartac,
  author = {Skatje Myers and Martha Palmer},
  title = {{ClearTAC: Verb Tense, Aspect, and Form Classification Using Neural Nets}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure