TL;DR
This paper introduces a statistical approach to automatically classify verbs into distinct semantic classes (unergative, unaccusative, object-drop) by analyzing their usage patterns in a large corpus. By leveraging features like transitivity, causativity, and animacy, the method achieves high accuracy, approaching human expert levels, and offers a more robust way to understand verb semantics than purely syntactic approaches.
What Problem Does This Paper Address?
Understanding the semantic nuances of verbs is crucial for many natural language processing tasks, from machine translation to question answering. Verbs can be grouped into classes based on their argument structure and the thematic roles they assign to their participants. For example, "run" (unergative) typically involves an agent performing an action, while "fall" (unaccusative) involves a theme undergoing a change of state. "Eat" (object-drop) can be transitive ("eat an apple") or intransitive ("eat").
Traditional approaches to verb classification often rely on hand-crafted rules or require extensive linguistic knowledge. This paper addresses the challenge of automatically acquiring these verb classes from raw text data, specifically by using statistical distributions of linguistic features observed in large corpora. The goal is to bridge the gap between syntax and semantics, showing that subtle differences in how verbs are used can reveal their underlying thematic structure.
Methods
The authors propose a statistical classification method that learns verb classes from corpus data. Their approach focuses on identifying features that reflect the thematic roles assigned by verbs. The core idea is that verbs belonging to different semantic classes will exhibit distinct patterns in their usage.
The key features they investigate are:
- Transitivity (TRANS): Whether a verb is used transitively (with a direct object) or intransitively.
- Causativity (CAUS): Whether a verb can be used to express causation (e.g., "The water boiled" vs. "I boiled the water").
- Animacy (ANIM): The tendency for the subject of a verb to be animate or inanimate.
- Passive Voice (PASS): The use of the passive voice, which is strongly correlated with transitive verbs.
- Past Participle Tag (VBN): The occurrence of the past participle form, often associated with passive or perfect constructions.
The method involves extracting these features from a large text corpus and then using statistical analysis to determine the distributional differences across verb classes. They then employ a machine learning classifier to predict verb class membership based on these features.
Datasets
The paper uses a large corpus for analysis. While the specific corpus is not explicitly named, it is described as a "large syntactically annotated corpus" and later as a "65-million word tagged ACL/DCI corpus."
For evaluation, they created a benchmark dataset consisting of 20 verbs from each of the three classes: unergatives, unaccusatives, and object-drops, totaling 60 verbs. This dataset was carefully selected to avoid verbs with massive departures from their intended sense and to include verbs that could occur in both transitive and passive constructions.
Key Insights and Ideas
- Thematic roles are statistically detectable: The paper strongly supports the idea that subtle differences in thematic role assignments are not just theoretical constructs but are reflected in observable statistical patterns in language use.
- Corpus statistics as a proxy for grammar: The research demonstrates that by analyzing frequency distributions of specific linguistic features, we can infer underlying grammatical and semantic properties of verbs, even without explicit linguistic annotation for those properties.
- Features beyond syntax are crucial: While transitivity is a key indicator, features like causativity and animacy of the subject provide additional discriminatory power, helping to distinguish between verb classes that might otherwise be confused based on syntax alone.
- Unaccusatives are harder to classify: The results consistently show that unergative verbs are the easiest to classify accurately, while unaccusatives and object-drop verbs present more challenges, suggesting they share more overlapping features.
- Passive voice is less informative than expected: Surprisingly, the passive voice feature (PASS) was found to be unhelpful, and sometimes even detrimental, to classification accuracy, contrary to initial hypotheses.
Key Results
The proposed statistical classification method achieved an overall accuracy of 69.5% using all five features. This is a significant improvement over baseline methods and is only slightly lower than the performance of human experts.
Class-specific accuracies highlight the difficulty in distinguishing between classes:
- Unergatives: 85% accuracy
- Unaccusatives: 63.2% accuracy
- Object-drops: 60.0% accuracy
The removal of the passive voice feature (PASS) actually improved performance slightly, while the removal of other features led to a 5-8% decrease in accuracy, indicating that each of the other features (TRANS, VBN, CAUS, ANIM) contributes meaningfully. The authors also found that more frequent verbs or classes were not necessarily more accurately classified.
Limitations and Future Work
- Ambiguous verb classifications: The current method struggles with verbs that can belong to multiple classes or exhibit different behaviors depending on context. The authors suggest that the concept of "intersective sets" and a finer-grained definition of semantic classes could address this.
- Distinguishing unaccusatives and object-drops: While the overall accuracy is respectable, the paper acknowledges that future work needs to focus on finding features that better discriminate between unaccusative and object-drop verbs.
- Handling ambiguous tokens: For future research, the authors plan to extend their method to classify individual verb tokens (specific instances of verb usage) rather than just verb types. This would involve combining corpus-based statistics with features of the specific usage instance.
- Interpreting other features: The paper notes that the role of VBN and PASS features requires further interpretation and analysis.
Summary updated