A Fully-Lexicalized Probabilistic Model for Japanese Zero Anaphora Resolution

Ryohei Sasano, Daisuke Kawahara, Sadao KurohashiCOLING 2008

TL;DR

This paper introduces a novel probabilistic model for resolving "zero anaphora" in Japanese, where pronouns are omitted but understood from context. By integrating automatically constructed case frames with a probabilistic framework that considers both explicit mentions and implicit zero pronouns, the model achieves improved resolution accuracy and efficiency.

What Problem Does This Paper Address?

Discourse analysis, a crucial component of natural language processing, often involves identifying how different mentions in a text refer to the same real-world entity (anaphora resolution). While English relies heavily on explicit pronouns (he, she, it), languages like Japanese frequently omit these pronouns, a phenomenon known as "zero anaphora." Resolving these omissions is challenging because the system must not only detect that a pronoun is missing but also identify what it refers to, all without explicit linguistic cues. Existing methods often treat zero pronoun detection and resolution separately, or rely on hand-crafted linguistic resources that are difficult to scale. This paper tackles the problem of jointly detecting and resolving zero anaphora in Japanese by proposing a unified probabilistic model.

Methods

The authors propose a probabilistic model that integrates case frame analysis with zero anaphora resolution. The core idea is to model the probability of a specific case frame and case assignment for a predicate, considering both explicit case components and potential zero pronouns.

Here's a breakdown of their approach:

  • Case Frame Construction: They leverage automatically constructed, wide-coverage case frames derived from a large Japanese corpus. These frames capture predicate-argument structures and are enhanced with generalized examples (e.g., categorizing nouns like "apple" as "FOOD") to mitigate data sparseness.
  • Joint Probability Model: The model estimates the probability of a case frame ($CF$) and a case assignment ($CA$) given a predicate ($v$), input case components ($ICC$), and existing entities ($ENT$). This is decomposed into:
  • $P(CF|v)$: The generative probability of a case frame.
  • $P(DCA, ICC|CF)$: The probability of direct case assignments (explicit mentions) given the case frame.
  • $P(ICA|ENT, CF, DCA)$: The probability of indirect case assignments (zero pronouns) given entities, the case frame, and direct assignments.
  • Direct vs. Indirect Case Assignment: The model distinguishes between direct case assignments (explicit noun phrases filling case slots) and indirect case assignments (zero pronouns filling case slots).
  • Salience Score: To prioritize potential antecedents, they introduce a salience score based on simple rules (e.g., mentions with "wa" topicalizer get higher scores, zero pronouns get a moderate score, and sentence beginnings reduce salience). This acts as a filter to reduce the search space.
  • Probabilistic Estimation: Probabilities are estimated from parsed corpora, case frames, and annotated data, using frequency counts and generalized categories/named entities.

Datasets

  • Training Data:
    • A large raw corpus (approx. 1.6 billion sentences) for constructing case frames.
  • A corpus of 80 million sentences for case structure analysis and estimating $P(nj)$ and $P(cj)$.
  • A hand-annotated anaphoric relation-tagged corpus of 186 web documents (979 sentences) for training and evaluation.
  • Test Data: 20 documents from the anaphoric relation-tagged corpus.

Key Insights and Ideas

  • Joint Detection and Resolution: The paper strongly advocates for treating zero pronoun detection and resolution as a single, integrated problem, rather than sequential steps. This allows the model to leverage the tight relationship between identifying a missing pronoun and finding its antecedent.
  • Leveraging Automatically Constructed Case Frames: By using wide-coverage, automatically generated case frames (enhanced with generalization), the model overcomes the limitations of manually created resources, making it more scalable and robust.
  • Probabilistic Framework for Implicit Information: The core innovation lies in extending probabilistic case frame analysis to handle implicit information (zero pronouns) by modeling the probability of indirect case assignments.
  • Salience as a Practical Filter: The introduction of a simple salience score mechanism effectively prunes unlikely antecedents, significantly speeding up the resolution process without sacrificing accuracy.

Key Results

  • The proposed model achieved an F-measure of 39.1% on web documents, which is competitive given the task's difficulty.
  • When using a salience score filter, the zero anaphora resolution process became approximately ten times faster.
  • Experiments showed that generalized categories improved F-measure, while generalized Named Entities contributed less, suggesting that common noun categories are more crucial for this task.
  • The parameter $\alpha$ (controlling the influence of entity content vs. surface case/location) was found to be most effective at $\alpha = 1/2$, achieving the highest F-measure.

Limitations and Future Work

  • Corpus Dependency: The performance is dependent on the quality and size of the training corpus, particularly the anaphoric relation-tagged data.
  • Web Document Specifics: The authors note that web documents can have less clear anaphoric relations compared to newspaper articles, which might explain lower absolute F-measures compared to some prior work on different domains.
  • Future Work: The authors plan to conduct larger-scale experiments and integrate their model into a fully-lexicalized probabilistic model for Japanese syntactic and case structure analysis. They also aim to explore more sophisticated methods for handling the sparseness of case slot examples.

Summary updated

BibTeX
@inproceedings{sasano2008fully,
  author = {Ryohei Sasano and Daisuke Kawahara and Sadao Kurohashi},
  title = {{A Fully-Lexicalized Probabilistic Model for Japanese Zero Anaphora Resolution}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2008},
}
Figure