A Latent Dirichlet Allocation Method for Selectional Preferences

Alan Ritter, Mausam, Oren EtzioniACL 2010 · OpenReview

Abstract

The computation of selectional preferences—the admissible argument values for a relation—is a well-known NLP task with broad applicability. We present LDA-SP, which utilizes LinkLDA to model selectional preferences. By simultaneously inferring latent topics and topic distributions over relations, LDA-SP combines the benefits of previous approaches: like traditional class-based approaches, it produces human-interpretable classes describing each relation's preferences, but it is competitive with non-class-based methods in predictive power.

We compare LDA-SP to several state-of-the-art methods, achieving an 85% increase in recall at 0.9 precision over mutual information. We also evaluate LDA-SP's effectiveness at filtering improper applications of inference rules, showing substantial improvement over existing systems.

Introduction

In Natural Language Processing (NLP), understanding selectional preferences is crucial for tasks like word sense disambiguation and semantic role labeling. For a relation like X is headquartered in Y, humans automatically infer that X should be an organization and Y should be a location.

Historically, researchers have used two main approaches:

  1. Class-based approaches: These rely on fixed taxonomies like WordNet. While interpretable, they are often brittle and struggle with out-of-vocabulary words or ambiguous senses.
  2. Similarity-based approaches: These utilize distributional statistics. They offer better coverage but function as "black boxes," lacking the interpretability of semantic classes.

LDA-SP aims to bridge this gap by using unsupervised topic modeling to learn latent semantic classes that are both broad in coverage and human-interpretable.

Methodology: The LDA-SP Model

The core of LDA-SP is the application of LinkLDA to relational data. In this framework, relations are treated as "documents" and the arguments within them as "words."

Graphical models for JointLDA (left) and LinkLDA (right). LDA-SP utilizes the LinkLDA structure to model argument pairs.

Unlike standard LDA, LinkLDA models the pair of arguments ($a_1, a_2$) for a relation. As shown in the graphical models above, LinkLDA allows the two arguments to have different topic IDs ($z_1, z_2$) while ensuring they are drawn from the same underlying per-relation distribution ($\theta$). This captures the semantic dependency between arguments—for instance, if the first argument is a "politician," the second is likely to be a "government body."

In contrast, JointLDA forces both arguments to share the exact same topic, which is often too restrictive for complex relations. IndependentLDA ignores the relationship between arguments entirely. LDA-SP finds the "sweet spot" by allowing flexibility while maintaining a shared thematic context.

Experimental Results

Predictive Power

We evaluated LDA-SP on a pseudo-disambiguation task, where the model must distinguish real argument pairs from random noise. The model was trained on a massive corpus of 2.4 million tuples extracted from the Web via TEXTRUNNER.

Precision-recall curves comparing LDA-SP against Jaccard and Mutual Information baselines.

As illustrated in the precision-recall curves, LDA-SP significantly outperforms traditional statistical methods. Specifically, it achieved an 85% increase in recall at 0.9 precision compared to mutual information methods. This demonstrates that the latent topics learned by LDA-SP capture selectional preferences more effectively than simple co-occurrence statistics.

Inference Rule Filtering

LDA-SP was also applied to filter inference rules from the DIRT database. For example, the rule "X defeats Y" $\implies$ "X plays Y" is only valid if X and Y are sports teams. LDA-SP successfully identifies these constraints, outperforming previous systems like Pantel et al. (2007) by providing a probabilistic framework that naturally handles the sparsity of relational data.

Conclusion

LDA-SP represents a significant step forward in modeling selectional preferences. By leveraging the generative power of LinkLDA, it provides a high-performance, unsupervised method for learning semantic constraints. The resulting latent topics are not only predictive but also semantically meaningful, with an 88% manual mapping accuracy to WordNet classes. Future work includes using LDA-SP to discover new inference rules and identify paraphrases automatically.

Summary updated

Abstract

The computation of selectional preferences, the admissible argument values for a relation, is a well-known NLP task with broad applicability. We present LDA-SP, which utilizes LinkLDA (Erosheva et al., 2004) to model selectional preferences. By simultaneously inferring latent topics and topic distributions over relations, LDA-SP combines the benefits of previous approaches: like traditional class-based approaches, it produces human-interpretable classes describing each relation's preferences, but it is competitive with non-class-based methods in predictive power. We compare LDA-SP to several state-of-the-art methods achieving an 85% increase in recall at 0.9 precision over mutual information (Erk, 2007). We also evaluate LDA-SP's effectiveness at filtering improper applications of inference rules, where we show substantial improvement over Pantel et al.'s system (Pantel et al., 2007).
BibTeX
@inproceedings{ritter2010latent,
  author = {Alan Ritter and Mausam and Oren Etzioni},
  title = {{A Latent Dirichlet Allocation Method for Selectional Preferences}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
  url = {https://openreview.net/forum?id=SJb2N2guWr},
}
Figure