Most traditional topic models, like Latent Dirichlet Allocation (LDA), treat themes as a flat list of independent categories. However, human knowledge is rarely organized this way. Information naturally follows a "general-to-specific" progression: a broad category like Science branches into Physics, which further narrows into Quantum Mechanics.
When models ignore this nested structure, they struggle to distinguish between high-level context and granular detail. Hierarchical topic modeling addresses this by discovering not just the content of topics, but the tree-like relationships between them.
The Bayesian Foundation: Growing Trees with nCRP
The challenge in building a topic hierarchy is that we rarely know the "shape" of the tree—how many levels it has or how many sub-topics exist under each branch—before looking at the data.
To solve this, David Blei, Thomas L. Griffiths, and Michael Jordan introduced the nested Chinese Restaurant Process (nCRP) in Hierarchical Topic Models and the Nested Chinese Restaurant Process.
Imagine a document as a tourist walking through a city of restaurants arranged in a tree. The tourist starts at the root (the most general topic) and chooses a table. Each table provides a path to a new restaurant (a more specific sub-topic) on the next level. This process allows the model to automatically discover the branching factor and depth of the hierarchy. By combining this with a hierarchical version of LDA (hLDA), researchers can move from "bag-of-words" to "path-through-a-tree," where words at the top of the tree are common across many documents (like "data" or "system"), while words at the leaves are highly specialized.
Scaling to the Deep Learning Era
While Bayesian non-parametrics provided the mathematical foundation, they often struggled with massive datasets. Modern research has shifted toward Neural Topic Models (NTMs) to gain scalability.
Ichiro Sakata introduced the Tree-Structured Neural Topic Model (TSNTM) to bridge this gap, using Variational Autoencoders (VAEs) to learn hierarchical latent spaces. To handle the "unknown shape" problem in a neural context, Ziye Chen and colleagues developed nTSNTM (Tree-Structured Topic Modeling with Nonparametric Neural Variational Inference), which uses a "stick-breaking" process to let the neural network determine the number of topics at each level without human intervention.
Another persistent issue in deep models is "latent variable collapse," where higher levels of the hierarchy fail to learn meaningful abstractions. Zhibin Duan and Xinyang Liu addressed this in Bayesian Progressive Deep Topic Model with Knowledge Informed Textual Data Coarsening Process by borrowing techniques from image generation. They "coarsen" the text data at different layers, providing intermediate supervision that forces the model to learn a logical progression from general themes to granular details.
From Generative Models to Embedding Spaces
As Large Language Models (LLMs) and embeddings became dominant, the focus shifted from purely generative statistical models to clustering in latent semantic spaces.
Traditional clustering often produces flat groupings. To fix this, Yu Meng and Jiaxin Huang developed TopClus (Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations). Instead of looking at word frequencies, TopClus projects documents into a spherical latent space where it performs recursive clustering to discover hierarchical taxonomies.
This embedding-centric approach allows for "seed-guided" discovery. In Discriminative Topic Mining via Category-Name Guided Text Embedding, the authors show that by providing just a few category names, models can navigate complex taxonomies by measuring "word specificity"—how much a term belongs to a narrow leaf versus a broad parent.
Why Hierarchy Matters in Practice
The move toward hierarchical modeling isn't just a theoretical preference; it solves specific failures in downstream NLP tasks:
- Summarization: Lea Frermann and Alexandre Klementiev demonstrated in Inducing Document Structure for Aspect-based Summarization that inducing latent hierarchical structure allows models to segment long documents and generate summaries focused on specific "aspects" (like "battery life" vs. "camera quality") rather than generic overviews.
- Sentiment Analysis: In the context of the COVID-19 "infodemic," Silvan Wehrli showed that hierarchical clustering is essential for distinguishing between broad public discourse and specific, conflicting sub-narratives, such as the difference between DIY mask-making and anti-mask protests.
- Retrieval-Augmented Generation (RAG): Modern RAG systems often struggle with complex queries. Kyungjae Lee argues in Learning to Explore and Select for Coverage-Conditioned Retrieval-Augmented Generation that without a hierarchical map of the information space, LLMs cannot effectively select or exclude specific thematic branches to satisfy nuanced user constraints.
By moving away from flat lists and toward structured trees, these models align more closely with the way information is naturally produced and consumed, enabling more precise navigation of massive, unstructured datasets.