Automatic Processing of Large Corpora for the Resolution of Anaphora References

Ido Dagan, Alon ItaiCOLING 1990

TL;DR

This paper proposes an automatic method for acquiring semantic constraints from large text corpora to resolve anaphora references and syntactic ambiguities. By analyzing co-occurrence patterns, the system learns which word combinations are semantically plausible, offering a more scalable and data-driven alternative to manually crafted selectional constraints.

What Problem Does This Paper Address?

Resolving ambiguities in natural language, particularly anaphora (e.g., figuring out what "it" refers to), often relies on "selectional constraints." These constraints define what kinds of words can grammatically and semantically fit together (e.g., a verb like "eat" typically takes an animate object). The major hurdle with traditional selectional constraints is the immense effort required to manually define them for broad domains like parliamentary proceedings. This paper tackles the challenge of acquiring these crucial semantic constraints automatically from large amounts of text, making them more practical for real-world applications.

Methods

The core of the paper's approach is to leverage statistical co-occurrence patterns from a large corpus as a proxy for semantic constraints. The process involves two main phases:

  1. Acquisition Phase:
    • Sentences from a large corpus (the Hansard corpus of Canadian parliamentary proceedings) are parsed using a PEG parser.
  2. A post-processing algorithm identifies syntactic relations like "subject-verb," "verb-object," and "adjective-noun."
    • Crucially, the system gathers statistics on which words co-occur in these specific syntactic relationships. For example, it counts how often "government" appears as the subject of "collect."
  3. Disambiguation Phase:
    • When an ambiguous construct (like a pronoun) is encountered, the system considers potential referents.
  4. For each potential referent, it checks if substituting it for the anaphor creates a co-occurrence pattern that has been observed frequently in the corpus.
  5. Candidates that produce frequent patterns are preferred, effectively acting as learned selectional constraints.

A key innovation is the reliance on statistical frequencies rather than pre-defined, manually curated rules. The authors also address the potential problem of circularity (using an un-disambiguated corpus to disambiguate) by noting that semantically valid patterns tend to occur far more frequently and thus can be identified even with some noise.

Datasets

The primary dataset used is the Hansard corpus, specifically a portion of it containing about 60 million words of Canadian parliamentary proceedings. For the experiment, a subset of 59 sentences containing the pronoun "it" was manually filtered and used for evaluation. The statistics were collected from a portion of the corpus totaling about 28 million words.

Key Insights and Ideas

  • Statistical Co-occurrence as Semantic Constraints: The paper demonstrates that frequent co-occurrence patterns in a large corpus can effectively capture semantic preferences, serving as a data-driven alternative to manually defined selectional constraints.
  • Scalability Advantage: Automatic acquisition from a corpus is significantly more scalable than manual knowledge engineering, making it feasible to cover broad domains.
  • Robustness to Ambiguity: Even with an un-disambiguated corpus, semantically valid patterns are often frequent enough to be reliably identified, mitigating the circularity problem.
  • Performance Beyond Traditional Constraints: In some cases, the statistical approach can resolve ambiguities that traditional selectional constraints alone cannot.

Key Results

The experiment focused on resolving references for the pronoun "it." * Coverage: The system achieved a "coverage" of 64% (38 out of 59 examples) where it could approve at least one candidate based on a threshold of 5 occurrences for alternative patterns. * Accuracy: For the cases where a candidate was approved, the "accuracy" was 87% (33 out of 38 cases). * Complete Resolution: In 18 of these cases, the statistical method uniquely identified the correct referent, completely resolving the ambiguity. This is a significant finding, as manual observation showed that traditional selectional constraints could only resolve these 18 cases in 7 instances.

Limitations and Future Work

  • Threshold Dependency: The system's performance is sensitive to the chosen threshold for co-occurrence frequency.
  • Limited Syntactic Relations: The experiment focused on a few specific syntactic relations ("subject-verb," "verb-object," "adjective-noun").
  • Manual Filtering: The initial selection of sentences for the experiment involved manual filtering, which is not scalable for a fully automatic system.
  • Future Work: The authors suggest that this statistical approach can be integrated with other disambiguation methods (syntactic, pragmatic) to further improve performance. They also note that while building a full-sized database for the entire corpus is resource-intensive, it is feasible for large-scale projects.

Summary updated

BibTeX
@inproceedings{dagan1990automatic,
  author = {Ido Dagan and Alon Itai},
  title = {{Automatic Processing of Large Corpora for the Resolution of Anaphora References}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1990},
}
Figure