Interactive Topic Model with Enhanced Interpretability

Jun Wang, Changsheng Zhao, Junfu Xiang, Kanji UchinoIUI Workshops 2019 · OpenReview

Introduction

Topic modeling is a cornerstone of text analysis, enabling the discovery of latent themes within large document collections. However, traditional methods like Latent Dirichlet Allocation (LDA) often produce unigram-based results that lack semantic clarity. For instance, a unigram model might group "apple" the fruit with "apple" the technology company, creating ambiguity. Researchers from Fujitsu and Columbia University address these limitations by introducing a phrase-based interactive topic modeling framework that prioritizes human interpretability and real-time steerability.

The Challenge of Unigram Models

Traditional "bag-of-words" models lose the contextual nuance provided by multi-word phrases (e.g., "neural networks" vs. "neural" and "networks"). While interactive models have been proposed to allow user intervention, they typically rely on computationally expensive structures like Dirichlet Forest priors, leading to high latency that hinders real-time interaction. This work bridges the gap by providing a semantically rich system that is efficient enough for fluid human-in-the-loop refinement.

Proposed Framework

The system utilizes a two-stage approach to integrate phrases into the topic modeling process.

The two-stage framework: phrase mining followed by interactive topic inference with a human feedback loop.
  1. Phrase Mining & Segmentation: The system first identifies meaningful n-grams using a data-driven phrase mining tool based on generalized suffix trees. Documents are then segmented into a combination of single words and identified phrases.
  2. PhraseLDA & Interactive Inference: An adaptation of LDA, termed "PhraseLDA," assigns topics to entire phrases.
  3. The Feedback Loop: User feedback is incorporated by updating asymmetric Dirichlet priors. This provides "pseudo-counts" that nudge the model to redistribute phrases during subsequent Gibbs sampling, rather than applying rigid hard-coded constraints.

Interactive Refinement

The framework supports several intuitive user operations to improve model alignment with domain knowledge:

  • Remove phrase/document: Discards irrelevant elements from a topic.
  • Split/Merge topics: Divides heterogeneous topics or combines redundant ones.
  • Add phrase: Injects missing concepts into the model.
Example of a 'split topic' operation where selecting a seed phrase like 'social media' helps separate distinct concepts.

A key finding is the "seed effect": users often only need to move a single representative phrase to trigger the automatic migration of related phrases, thanks to the model's understanding of word co-occurrence.

Evaluation and Results

The model was evaluated on a dataset of 19,852 machine learning papers from ICML, NIPS, and arXiv. This technical corpus is rich in multi-word phrases like "stochastic gradient descent," making it an ideal testbed.

Key results include:

  • Performance: The use of asymmetric priors achieved a 50x speedup in convergence latency compared to previous interactive methods, enabling a truly "live" experience.
  • User Experience: In a study with five participants, all users reported significant improvements in topic coherence. The "split topic" and "remove phrase" operations were identified as the most valuable tools for refinement.
  • Interpretability: Phrase-based representations were found to be inherently more readable than unigram lists, as phrases serve as more natural "units of meaning" for human cognition.

Conclusion and Future Work

This phrase-based approach represents a significant advancement in making topic models accessible and controllable for non-experts. While the current system is highly efficient, future work will explore sparsity-aware inference to handle even larger datasets and develop new standardized metrics specifically designed for phrase-based topic evaluation.

Summary updated

BibTeX
@inproceedings{wang2019interactive,
  author = {Jun Wang and Changsheng Zhao and Junfu Xiang and Kanji Uchino},
  title = {{Interactive Topic Model with Enhanced Interpretability}},
  booktitle = {IUI Workshops},
  year = {2019},
  url = {https://openreview.net/forum?id=kZhkEdkmvq},
}
Figure