Abstract: Topics generated by topic models are typically represented as list of terms. To reduce the cognitive overhead of interpreting these topics for end-users, we propose labelling a topic with a succinct phrase that summarises its theme or idea. Using Wikipedia document titles as label candidates, we compute neural embeddings for documents and words to select the most relevant labels for topics. Comparing to a state-of-the-art topic labelling system, our methodology is simpler, more efficient and finds better topic labels.
Introduction
Topic models like Latent Dirichlet Allocation (LDA) are indispensable for navigating massive document collections. However, their standard output—a "bag of words"—often imposes a significant cognitive burden on users. For instance, a list like blogs, vmware, server, virtual, oracle requires manual interpretation to identify the underlying theme of "Virtualization."
This paper introduces NETL (Neural Embedding Topic Labelling), a streamlined approach to automatic topic labelling. By leveraging Wikipedia article titles as candidate labels and mapping them into a shared vector space with topic terms using neural embeddings, NETL produces succinct, human-readable summaries that outperform complex, search-engine-dependent baselines.
Methodology: Neural Embedding Topic Labelling
The NETL framework operates in two distinct phases: candidate generation and supervised re-ranking.
Candidate Generation and the Embedding Dichotomy
The system uses English Wikipedia as its primary knowledge base. It represents both Wikipedia titles and topic terms as vectors. A crucial innovation is the use of two complementary embedding techniques:
- word2vec (Skip-gram): Treats labels as single tokens, capturing broad contextual relationships and favoring generic or abstract labels.
- doc2vec (DBOW): Captures compositional semantics, tending to identify more specific, fine-grained concepts.
As shown in the table above, word2vec might suggest "Software" for a virtualization topic, while doc2vec identifies "Microsoft Visual Studio." Combining these approaches ensures a diverse and high-quality candidate pool.
Supervised Re-ranking
To select the best label from the candidates, NETL employs a Support Vector Regression (SVR) model. This model integrates several features including LetterTrigram (character-level overlap), PageRank (concept centrality in Wikipedia), TopicOverlap (direct word matches), and NumWords (label length).
Experimental Setup
The authors evaluated NETL across four diverse domains: general blogs (BLOGS), books (BOOKS), news articles (NEWS), and biomedical abstracts (PUBMED). Human gold-standard ratings (on a 0–3 scale) were collected via CrowdFlower to provide a robust ground truth for evaluation.
Key Results and Performance
NETL consistently demonstrates superior performance over the previous state-of-the-art system, LGNB. The improvements are particularly striking in specialized domains like PUBMED, suggesting that general-purpose knowledge bases like Wikipedia, when queried via embeddings, can effectively label expert-level content without requiring domain-specific experts.
As illustrated in the boxplots, NETL (blue) maintains a higher median rating and tighter distribution than LGNB (red) across nearly all metrics and datasets. Analysis of the "upper bound" scores further confirms that NETL's candidate generation phase is fundamentally more effective at identifying high-potential labels than previous methods.
Conclusion and Future Directions
NETL offers a simpler, faster, and more accurate alternative to traditional topic labelling systems. By moving away from fragile external search APIs and toward pre-computed embeddings, it provides a deployable solution for real-world applications. Future research may explore integrating Wikipedia's hierarchical category structure or optimizing for "interestingness" to further enhance the utility of generated labels for end-users.
Summary updated