Multi Path Hierarchical Topic Modeling

This direction is merged into another direction: tree-structured hierarchy constraint. Its last version stays readable and citable here.

Version 1 of 111 papers21 conceptsfrozen 2026-10-04

Traditional topic models like Latent Dirichlet Allocation (LDA) treat documents as "bags of words" that belong to a flat mixture of topics. However, human knowledge is naturally hierarchical: a document about "Deep Learning" is also about "Machine Learning" and "Computer Science." To capture this, researchers developed hierarchical topic models that organize information from general concepts at the root to specific details at the leaves.

The evolution of this field centers on a single, critical tension: how to allow a document to navigate this hierarchy without being trapped in a single branch.

The Single Path Constraint

The foundational approach to learning these hierarchies was The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. In this work, David Blei, Thomas L. Griffiths, and Michael Jordan introduced the Hierarchical LDA (hLDA) model.

The hLDA model uses a stochastic process called the Nested Chinese Restaurant Process (nCRP) to discover a tree structure where each node is a topic. However, hLDA imposes a strict constraint: each document must choose exactly one path from the root to a single leaf. While this successfully captures the "general-to-specific" transition, it fails for multi-faceted documents. A news article about the economic impact of a sports stadium cannot easily exist in a model that forces it to choose between the "Sports" branch and the "Economics" branch.

From Trees to Directed Acyclic Graphs

To break this rigidity, researchers moved toward more flexible structures. In Mixtures of hierarchical topics with Pachinko allocation, David Mimno and Andrew McCallum introduced the Pachinko Allocation Model (PAM).

Instead of a simple tree, PAM uses a Directed Acyclic Graph (DAG). This allows topics to be correlated in complex ways; a sub-topic (like "Statistical Significance") can be a child of multiple high-level topics (like "Biology" and "Psychology"). Later, Wei Li 0121, McCallum, and David M. Blei extended this in Nonparametric Bayes Pachinko Allocation, using a variant of the Hierarchical Dirichlet Process (HDP) to automatically learn the number of topics and their correlations. By allowing documents to mix multiple paths through the graph, these models better reflect the "thematic borrowing" seen in real-world text.

Word Level Path Assignment

The most significant leap in flexibility came from shifting the "path decision" from the document level to the word level. In A Nested HDP for Hierarchical Topic Models and its subsequent refinement in Nested Hierarchical Dirichlet Processes, John Paisley, Chong Wang, and David Blei introduced the Nested HDP (nHDP).

The intuition behind nHDP is that while a document as a whole might relate to a general area, its individual words are the ones that actually reside at different levels of specificity and in different thematic branches. In nHDP:

  1. A global tree of topics is shared across all documents.
  2. Each document defines its own distribution over the paths in that tree.
  3. For every word in the document, a specific path is sampled, and then a node (topic) along that path is selected.

This mechanism allows a single document to "borrow" concepts from entirely different branches of the global hierarchy simultaneously. It solves the sports-economics dilemma by allowing "stadium" to come from a sports path and "revenue" to come from an economics path, all while maintaining the hierarchical relationship to their respective general roots.

Granularity and Discourse

This multi-path philosophy has been extended to capture even finer nuances of text. Jordan Lee Boyd-Graber and Philip Resnik applied these ideas to supervised settings in Lexical and Hierarchical Topic Regression. By allowing individual sentences to traverse different paths, they could distinguish between the "agenda" (the topic hierarchy) and the "framing" (how specific words are used to express sentiment or ideology within those topics).

More recently, the challenge of hierarchical modeling has moved to semi-structured data. In WebDP: Understanding Discourse Structures in Semi-Structured Web Documents, Peilin Liu and Xianpei Han point out that modern web pages often lack a single overarching topic entirely. This reinforces the necessity of the multi-path approach: when documents are fragmented and multi-thematic, the model must be able to represent multiple independent roots and divergent semantic trajectories to truly understand the discourse.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_multi_path_hierarchical_topic_modeling_24c53d5b,
  title = {Multi Path Hierarchical Topic Modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-path-hierarchical-topic-modeling/rdr_24c53d5bf08a4520939e9b25f20fef62},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_multi_path_hierarchical_topic_modeling_24c53d5b_v1,
  title = {Multi Path Hierarchical Topic Modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multi-path-hierarchical-topic-modeling/rdr_24c53d5bf08a4520939e9b25f20fef62/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}