TL;DR
This paper tackles the challenge of "notional anaphora," where pronouns disagree in number with their antecedents for semantic reasons (e.g., "the government... they"). The authors build a predictive model using the large OntoNotes corpus, achieving 86.81% accuracy in predicting notional anaphora, a significant improvement over baseline methods. Their findings highlight the crucial role of genre, document position, and semantic entity types in this phenomenon.
What Problem Does This Paper Address?
In English, pronouns typically agree in number with their antecedents (e.g., "the company... it"). However, a fascinating phenomenon called "notional agreement" or "notional anaphora" occurs where this grammatical agreement is overridden by semantic considerations. A prime example is when a singular noun referring to a group (like "the government" or "the committee") is referred to by a plural pronoun ("they"). This poses a significant challenge for natural language processing (NLP) tasks like coreference resolution and natural language generation, as it deviates from strict grammatical rules. Previous research has primarily focused on subject-verb agreement in this context, with limited corpus-based studies on pronoun agreement due to a lack of adequately annotated data. This paper aims to bridge this gap by developing a robust predictive model for notional anaphora in English.
Methods
The authors employ an ensemble of decision trees, specifically the Extra Trees algorithm, to build their predictive model. This approach was chosen over deep learning due to the limited size of the available annotated data for this specific phenomenon. They extract a rich set of features, including:
- Syntactic and Lexical Features: POS tags and dependency functions for the anaphor, antecedent, and their governing tokens.
- Entity and Genre Features: Entity types (e.g., PERSON, ORGANIZATION) and genre/modality of the text.
- Positional and Distance Features: Distance in tokens between the anaphor and antecedent, and the position of the anaphor within the document (expressed as a percentile).
- Semantic Features: VerbNet classes for governing verbs and a custom "generic" feature for phrases like "anyone."
A grid search with 5-fold cross-validation was used to find optimal hyperparameters, and the model was trained on 90% of the data, with the remaining 10% reserved for testing.
Datasets
The primary dataset used is the OntoNotes corpus (Version 5), which is the largest manually annotated corpus for coreference in English, containing approximately 1.59 million tokens with coreference annotations. The authors filtered this corpus to extract anaphor-antecedent pairs where a singular NP is referred to by a plural pronoun, and where the antecedent's lexical head is known to exhibit notional agreement. This resulted in a dataset of 3,488 anaphor-antecedent pairs, with 1,209 exhibiting notional agreement.
Key Insights and Ideas
- Genre Matters Significantly: The study reveals that genre plays a crucial role in notional anaphora. Spoken genres (like
bc.conv) show a much higher frequency of notional agreement compared to written genres, especially translations. - Document Position is Predictive: The position of the anaphor within a document is a strong predictor. Notional agreement becomes more frequent in the latter half of documents, suggesting potential "speaker fatigue" or editorial effects.
- Entity Type is Key: The semantic category of the antecedent is highly influential. "ORGANIZATION" and "QUANTITY/TIME" entities show a preference for notional agreement, while "PERSON" entities are more complex, with some (like "family") tending towards strict agreement, and others potentially allowing for Type III (gender-neutral) notional agreement.
- Not All Notional Cases are Equal: The paper distinguishes between different types of notional anaphora (Type I/II for collective/distributive nouns and Type III for gender-neutral pronouns), noting that their model handles Type III cases reasonably well.
- Predictability Exists Despite Apparent Unpredictability: While notional agreement can seem arbitrary, the model demonstrates that a significant degree of predictability can be achieved by considering a combination of linguistic and contextual features.
Key Results
The developed ensemble classifier achieved an accuracy of 86.81% in predicting notional agreement on the test set. This represents a substantial improvement of over 20% compared to a majority baseline that would always guess "strict agreement" (which has a 65.6% accuracy). The confusion matrix shows that the classifier is particularly good at avoiding misclassifying strict agreement cases as notional, though it does misclassify about one-third of actual notional cases as strict. The most important features identified were the anaphor's person (1st vs. 3rd), the POS tag of the anaphor's governor, and genre.
Limitations and Future Work
- Data Limitations: While OntoNotes is large, the number of notional anaphora examples is still relatively small (around 1,200 pairs), which limits the feasibility of more complex models like deep learning. The exclusion of indefinite NPs as antecedents also means certain cases are not covered.
- Feature Engineering: The authors acknowledge that some features, like VerbNet classes, are not optimal for capturing semantic nuances. They suggest exploring more advanced semantic representations, such as cluster-based or vector space models, or even trainable embeddings.
- Ambiguity and Acceptability: The classifier sometimes defaults to "strict agreement" for ambiguous cases where either form might be acceptable. Future work could involve human acceptability studies to better understand these borderline cases.
- Broader NLP Integration: The authors suggest that their model's probability estimates could be valuable for state-of-the-art coreference resolution systems and other NLP applications that deal with anaphoric resolution.
Summary updated