Automatic Labelling of Topics with Neural Embeddings

Shraey Bhatia, Jey Han Lau, Timothy Baldwin

Abstract

Topics generated by topic models are typically represented as lists of terms. To reduce the cognitive overhead of interpreting these topics for end-users, we propose labelling a topic with a succinct phrase that summarises its theme or idea. Using Wikipedia document titles as label candidates, we compute neural embeddings for documents and words to select the most relevant labels for topics. Compared to a state-of-the-art topic labelling system, our methodology is simpler, more efficient, and finds better topic labels.

Introduction

Topic models are powerful tools for uncovering hidden themes in massive document collections, but they suffer from a significant "user experience" problem. Traditionally, a topic is presented to a human as a list of its top-10 most probable words (e.g., school, student, university, teacher). While accurate, this requires high cognitive load for users to mentally synthesize these words into a coherent concept.

Previous attempts to automate labelling relied on complex Information Retrieval (IR) pipelines, often requiring live queries to external search APIs. These methods are frequently brittle, slow, and fail on domain-specific data. We introduce NETL (Neural Embedding Topic Labelling), a system that leverages pre-trained word and document embeddings to create a simpler, more efficient, and more accurate labelling pipeline.

The NETL Framework

The NETL framework operates in two primary stages:

  1. Candidate Generation: Instead of live-searching, we pre-compute distributed representations (embeddings) for all English Wikipedia article titles. We use doc2vec (DBOW) to capture compositional semantics and word2vec (Skip-gram) to capture contextual relationships. A topic's embedding is derived from its top-10 terms, and the system identifies the top-100 Wikipedia titles most similar to the topic based on cosine similarity.
  2. Candidate Ranking: Once candidates are identified, we use a supervised Support Vector Regression (SVR) model to select the best label. The ranker utilizes features including character-level overlap (LetterTrigram), PageRank (centrality), label length, and lexical overlap.

The Embedding Dichotomy

A key insight of this research is that doc2vec and word2vec "prefer" different types of labels. Doc2vec tends to find fine-grained, specific concepts, while word2vec finds more abstract, generic labels. Summing their scores allows the system to balance specificity and abstraction effectively.

Comparison of labels generated by word2vec and doc2vec for a sample topic, illustrating the difference between abstract and specific descriptors.

Experimental Results

We evaluated NETL against the previous state-of-the-art (LGNB) across four distinct datasets: BLOGS, BOOKS, NEWS, and PUBMED (biomedical abstracts).

Performance comparison of NETL vs. LGNB across four test domains and various training configurations, showing superior accuracy and consistency.

As shown in the results, NETL outperformed LGNB across almost all metrics. In the highly specialized PUBMED domain, NETL achieved a Top-1 average rating of 1.99 compared to LGNB’s 1.79, proving its effectiveness in technical domains. Furthermore, the "Upper Bound" performance was consistently higher for NETL, indicating that the neural embedding approach is superior at surfacing high-quality candidates in the first place.

Conclusion

NETL demonstrates that moving away from external search APIs toward pre-computed neural embeddings makes topic labelling faster and more robust without losing accuracy. Even for highly specialized domains, Wikipedia titles proved to be excellent label candidates when paired with neural similarity measures. Future work will explore integrating Wikipedia's hierarchical category structure to further improve label precision and "interestingness."

Summary updated

Abstract

Topics generated by topic models are typically represented as list of terms. To reduce the cognitive overhead of interpreting these topics for end-users, we propose labelling a topic with a succinct phrase that summarises its theme or idea. Using Wikipedia document titles as label candidates, we compute neural embeddings for documents and words to select the most relevant labels for topics. Compared to a state-of-the-art topic labelling system, our methodology is simpler, more efficient, and finds better topic labels.
BibTeX
@inproceedings{bhatia2016automatic,
  author = {Shraey Bhatia and Jey Han Lau and Timothy Baldwin},
  title = {{Automatic Labelling of Topics with Neural Embeddings}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2016},
  url = {https://openreview.net/forum?id=mt9fpgynIRV},
}

Loading related content...

Figure