A Probabilistic Disambiguation Method Based on Psycholinguistic Principles

Hang Li

TL;DR

This paper introduces a novel probabilistic method for resolving syntactic ambiguity in natural language processing by integrating psycholinguistic principles (Lexical Preference Rule, Right Association Principle, Attach Low and Parallel Principle) with a probabilistic framework. The authors propose a "back-off" method to combine lexical and syntactic likelihoods, achieving a significant 89.2% accuracy on disambiguation tasks.

What Problem Does This Paper Address?

Syntactic disambiguation, the process of determining the correct grammatical structure of a sentence when multiple interpretations are possible, remains a central challenge in natural language processing (NLP). While psycholinguistics has proposed several principles (like the Lexical Preference Rule and Right Association Principle) to explain how humans resolve these ambiguities, effectively implementing these principles within computational models has been difficult. Existing probabilistic methods, while powerful, often struggle to fully capture human-like disambiguation. This paper aims to bridge this gap by developing a probabilistic approach that explicitly incorporates these well-established psycholinguistic principles.

Methods

The core of the proposed method lies in a probabilistic framework that operationalizes psycholinguistic principles:

  • Lexical Preference Rule (LPR): This is modeled using a "three-word probability." The authors define this as the conditional probability of a word appearing in a specific case slot given the verb and the noun in that slot. The "lexical likelihood" of an interpretation is then calculated as the geometric mean of these three-word probabilities. This captures the idea that certain word co-occurrences are more semantically consistent than others.
  • Right Association Principle (RAP) and Attach Low and Parallel Principle (ALPP): These syntactic principles are implemented using a "length probability." This probability is derived from the frequency of attachments in correct syntactic trees, considering the "length" (number of words) of the constituent categories involved in the attachment. This captures the tendency to attach new phrases to nearby existing ones (RAP) and to prefer parallel structures (ALPP). The "syntactic likelihood" is the geometric mean of these length probabilities.
  • Back-Off Method: To combine the lexical and syntactic information, the authors propose a "back-off" strategy. They first use the lexical likelihood. If the lexical likelihoods of two interpretations are very close (within a threshold), they then use the syntactic likelihood to make the final decision. This reflects the idea that lexical preferences often take precedence, but syntactic structure becomes important when lexical cues are ambiguous.

Datasets

  • Training Data:
    • 181,250 case frames extracted from the WSJ (Wall Street Journal) bracketed corpus of the Penn Tree Bank.
  • 963 sentences from the WSJ tagged corpus of the Penn Tree Bank, with correct syntactic trees selected manually.
  • Test Data:
    • 249 sentences from a part of the WSJ tagged corpus not used in training.

The paper does not introduce new benchmarks but utilizes existing, widely recognized corpora for NLP research.

Key Insights and Ideas

  • Unifying Psycholinguistics and Probability: The paper demonstrates that psycholinguistic principles, often seen as qualitative heuristics, can be effectively formalized and integrated into a quantitative probabilistic framework for NLP tasks.
  • "Length Probability" for Syntactic Principles: The novel concept of "length probability" provides a concrete way to operationalize abstract syntactic principles like RAP and ALPP, moving beyond simple counts of attachments.
  • "Back-Off" as a Hierarchical Decision Strategy: The back-off method effectively models the observed human tendency to prioritize lexical information before resorting to syntactic cues when resolving ambiguity. This is a more nuanced approach than simply averaging or weighting different types of evidence.
  • Data Sparseness Mitigation: While not fully resolved, the proposed methods for estimating probabilities (e.g., smoothing with word classes, though not detailed in this excerpt) and the use of both three-word and two-word probabilities help address the data sparseness problem inherent in estimating probabilities from limited data.

Key Results

  • The proposed method, combining lexical likelihood (based on three-word probabilities) and syntactic likelihood (based on length probabilities) using a back-off strategy, achieved an accuracy of 89.2% in resolving prepositional phrase attachment and coordinate structure ambiguities.
  • This accuracy is presented as "number 1 accuracy," meaning the correct interpretation was ranked first among the candidates.
  • A breakdown of the results shows that the lexical component (Lex3) contributed significantly (95.7% accuracy on its own and accounted for 47.0% of disambiguation decisions), followed by the two-word lexical likelihood (Lex2) and then the syntactic likelihood (Syn).
  • The paper also compares its probabilistic approach to a deterministic implementation of RAP, finding the probabilistic method to be superior in ranking interpretations.
  • Using a PCFG model for syntactic preference yielded a slightly lower accuracy (86.7%) compared to the length probability model.

Limitations and Future Work

  • Data Sparseness: While the methods aim to mitigate data sparseness, it is acknowledged that it cannot be completely resolved. The authors suggest that increasing the amount of training data, particularly for lexical preference, would further improve results.
  • Ambiguity Types: The experiments focused primarily on prepositional phrase attachment and coordinate structure ambiguities. The authors suggest that their approach could be extended to other types of ambiguities.
  • Longer Sentences: The improvement from syntactic likelihood was noted as less significant for sentences with only two interpretations. The authors expect the effect of syntactic likelihood to become more pronounced with longer and more complex sentences, which they plan to investigate in future work.
  • Stochastic vs. Deterministic: While the stochastic approach outperformed the deterministic one in their experiments, the improvement was not significant. Further analysis with longer sentences is suggested to better highlight this difference.

Summary updated

Abstract

Hang Li. Fourth Workshop on Very Large Corpora. 1996.
BibTeX
@inproceedings{li1996probabilistic,
  author = {Hang Li},
  title = {{A Probabilistic Disambiguation Method Based on Psycholinguistic Principles}},
  booktitle = {VLC @ COLING},
  year = {1996},
  url = {https://openreview.net/forum?id=qEHFujc5BjM},
}
Figure