TL;DR
This paper tackles the challenge of "bridging resolution," a linguistic phenomenon where an anaphoric expression refers to something not directly mentioned but inferentially related. The authors found that existing rule-based methods, successful on other corpora, perform poorly on the ARRAU corpus due to a mismatch in how "bridging" is annotated, specifically the prevalence of "lexical bridging" in ARRAU. They adapt the rule-based system to handle this, achieving competitive results against a learning-based approach.
What Problem Does This Paper Address?
Bridging resolution is a crucial task in Natural Language Processing (NLP) for understanding discourse. It involves linking anaphoric noun phrases (like "the door" in "I saw a house. The door was red.") to their antecedents, where the relationship isn't one of direct identity but rather an implicit connection (e.g., a door is part of a house). While significant progress has been made in coreference resolution (linking identical referents), bridging resolution remains more challenging due to the inferential nature of the links. This paper specifically addresses the performance of existing bridging resolution systems on the ARRAU corpus, a multi-domain dataset designed for this task. The authors highlight a critical issue: the ARRAU corpus contains a substantial amount of "lexical bridging" (based on word-level semantic relations like meronymy) which differs from the "referential bridging" (based on discourse-level relationships) that many existing systems are optimized for.
Methods
The paper presents two main approaches:
- Re-implementation and Adaptation of a Rule-Based System: The authors start by re-implementing the rule-based system proposed by Hou et al. (2014), which achieved state-of-the-art results on the ISNotes corpus. They then adapt this system to better handle the ARRAU corpus. The key adaptation involves recognizing and incorporating rules for "lexical bridging," which was a significant component of the ARRAU data but not adequately addressed by the original system. The adapted system retains some of the original rules while adding new, more specific ones to increase precision and recall.
- Learning-Based Method: A Support Vector Machine (SVM) classifier is implemented using a pair-based approach, similar to Soon et al. (2001). This classifier is trained on pairs of potential anaphor-antecedent candidates. The system is provided with the same information as the rule-based system and uses a rich set of features, including markable features (word properties, grammatical roles, etc.) and pair features (distance, head match, WordNet relations, etc.).
Datasets
The primary dataset used is the ARRAU corpus. This corpus is multi-domain, containing text from three sources:
- RST (newspaper)
- TRAINS (dialogue)
- PEAR (narratives)
The authors also mention the ISNotes corpus as the dataset on which the Hou et al. (2014) system was originally developed and evaluated.
Key Insights and Ideas
- The ARRAU Corpus's Unique Annotation: The most significant insight is the discovery that ARRAU heavily features "lexical bridging" (e.g., "house" and "door" as a meronymy pair) rather than solely "referential bridging" (where an anaphor refers to a previously mentioned entity indirectly). This distinction is crucial for understanding why existing systems fail.
- Lexical vs. Referential Bridging: The paper clearly distinguishes between these two types of bridging, explaining that referential bridging is about discourse-level interpretation, while lexical bridging is about word-level semantic relationships. This distinction is key to adapting the rule-based system.
- Rule-Based System Adaptability: The study demonstrates that a rule-based system, when appropriately adapted to the specific characteristics of a corpus (like ARRAU's lexical bridging), can achieve competitive performance.
- Rule-Based Outperforms Learning-Based: Surprisingly, the adapted rule-based system consistently outperformed the learning-based approach across various tasks and domains within the ARRAU corpus.
Key Results
- The re-implemented Hou et al. (2014) rule-based system achieved a very low F1 score of 0.3 on the RST portion of the ARRAU corpus, a stark contrast to its performance on ISNotes.
- The adapted rule-based system achieved a reasonable performance, specifically 39.8% accuracy for bridging anaphora resolution (finding an antecedent for a given bridging anaphor) on the RST domain.
- The adapted rule-based system outperformed the learning-based system in all evaluated settings.
- The performance of the adapted rule-based system was even better on the PEAR and TRAINS domains than on the RST domain, though these datasets are smaller.
Limitations and Future Work
- Small Dataset Size: The authors acknowledge that the improved performance on the PEAR and TRAINS domains might be due to their smaller size, suggesting that these results should be interpreted with caution.
- Internal vs. Official Scorer: The paper notes that their internal evaluation metrics might differ from the official scorer used for the shared task, potentially leading to lower official scores due to the handling of special cases.
- Future Work: The paper implicitly suggests that further research could focus on more sophisticated methods for distinguishing between lexical and referential bridging automatically, as well as exploring more advanced learning-based architectures that can better capture the nuances of lexical bridging. The availability of the system code (mentioned in a footnote) suggests a path for future research and development.
Summary updated