A Two-Dimensional Topic-Aspect Model for Discovering Multi-Faceted Topics

Michael J. Paul, Roxana GirjuAAAI 2010 · DOI

Abstract

Traditional topic models like Latent Dirichlet Allocation (LDA) primarily capture a single dimension of topicality, often overlooking secondary layers such as perspective or sentiment. We present the Topic-Aspect Model (TAM), a Bayesian mixture model designed to jointly discover topics and aspects. By incorporating hidden variables for level (background vs. topical) and route (aspect-neutral vs. aspect-dependent), TAM effectively untangles intersecting dimensions in text. Evaluations on scientific abstracts and political editorials demonstrate TAM's superior ability to identify multi-faceted themes and its robustness in low-data scenarios compared to standard LDA and ccLDA.

Introduction

Standard probabilistic topic models, most notably Latent Dirichlet Allocation (LDA), typically organize text along a single dimension: topicality. While effective, this ignores the reality that language often carries secondary layers like sentiment, perspective, or disciplinary focus.

Previous attempts to solve this usually forced a choice: a word was either "topical" or "perspectival," but rarely both. For example, in a corpus of scientific papers, a topic like "Grammar" might be discussed from a mathematical perspective (focusing on formal languages) or a linguistic perspective (focusing on syntax and prosody). Existing models struggled to capture how these perspectives manifest within specific topics. The authors aim to provide a model that can automatically learn these intersecting dimensions without requiring manual document labels.

The Topic-Aspect Model (TAM)

The authors introduce TAM (Topic-Aspect Model). Its core innovation is a generative process where every word token is associated with four hidden variables:

  1. Topic ($z$): The general subject matter.
  2. Aspect ($y$): The underlying theme or perspective.
  3. Level ($\ell$): A binary switch distinguishing between "background" (common) words and "topical" words.
  4. Route ($x$): A binary switch determining if the word is "aspect-neutral" (shared across perspectives) or "aspect-dependent" (specific to one perspective).
Plate notation for the Topic-Aspect Model (TAM) showing the dependencies between hidden variables.

Unlike its predecessor ccLDA, which requires documents to be pre-grouped into collections, TAM can be entirely unsupervised, discovering aspects from a single mixed collection. The model uses Gibbs sampling for inference, iteratively estimating these hidden variables to find the most likely structure of the data.

Experimental Setup

The researchers evaluated TAM on three primary datasets:

  • CL-Only: Abstracts from major Computational Linguistics conferences (ACL, COLING, HLT) published after 1980.
  • CL-LING: A combination of the CL abstracts and abstracts from four major Linguistics journals (e.g., Language, Linguistic Inquiry).
  • I-P Dataset: A corpus of 594 editorials written by Israeli and Palestinian authors regarding their conflict, providing a clear ground truth for "perspectives."

Results and Discussion

Qualitative Analysis

By modeling the intersection of topic and aspect, TAM can show how the same topic changes flavor. For instance, in the "Lexical Semantics" topic, the computational aspect highlights clustering and distributional methods, while the linguistic aspect highlights ontology and conceptual frameworks. TAM applies the aspect dimension to both background words and topical words. This is crucial because even common "filler" words can vary by perspective (e.g., an Israeli editorial might frequently use the word "settlement" while a Palestinian one uses "occupation" as a background term).

Example of TAM's two-dimensional word distributions across background and topical levels.

Quantitative Performance

In a task to predict an author's perspective (Israeli vs. Palestinian), TAM's learned aspect distributions were highly predictive. With only 1% of the corpus used for training, TAM achieved 70.3% accuracy, compared to just 53.3% for LDA. Even with 80% training data, TAM maintained a slight edge. Furthermore, using "word intrusion" tests, TAM performed slightly better than ccLDA and significantly better than standard LDA, proving that its multi-dimensional constraints don't degrade topic quality.

Conclusion

While the model is powerful, interpreting results in political corpora can be subtle due to overlapping aspects. The model also introduces more hyperparameters than LDA, increasing computational cost. Future directions include adapting TAM for multi-perspective document summarization or more nuanced sentiment analysis where "topic" and "sentiment" are the two primary dimensions.

Summary updated

BibTeX
@inproceedings{paul2010two,
  author = {Michael J. Paul and Roxana Girju},
  title = {{A Two-Dimensional Topic-Aspect Model for Discovering Multi-Faceted Topics}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2010},
  doi = {10.1609/aaai.v24i1.7669},
}
Figure