Joint Topic Modeling and Network Representation Learning

This direction is merged into another direction: topic-aware link prediction. Its last version stays readable and citable here.

Version 1 of 142 papers53 conceptsfrozen 2026-10-04

Traditional topic models like Latent Dirichlet Allocation (LDA) treat documents as "islands"—isolated bags of words that exist independently of one another. However, in the real world, documents are almost always part of a larger ecosystem. Scientific papers are connected by citations, web pages by hyperlinks, and social media posts by follower networks.

The research direction represented here moves beyond isolated text analysis to jointly model document content and network topology. By treating the links between documents not just as metadata, but as primary semantic signals, these models produce more coherent topics and more accurate representations of the entities within a network.

The fundamental intuition behind this research is that connectivity provides a powerful regularizer for language. If two documents are linked, they likely share a thematic relationship, even if they use different vocabularies (e.g., one paper uses "deep learning" while its cited reference uses "multilayer perceptrons").

In the foundational Relational Topic Models for Document Networks, Jonathan Chang and David M. Blei demonstrated that conditioning link probabilities on the latent topic assignments of words allows the model to identify themes that specifically explain why connections form. Similarly, Zhen Guo and colleagues proposed the Citation-Topic (CT) model, which views a document’s content as a mixture of its own unique ideas and the topics of the documents it references. This approach uses the network as a "semantic anchor," preventing the model from being led astray by noisy or infrequent words in individual documents.

The Reciprocal Feedback Loop

A key advantage of joint modeling is the bidirectional flow of information. Textual content helps disambiguate sparse network structures, while network structure helps resolve linguistic ambiguity.

  1. Text Disambiguates Links: In social networks, a "link" can mean many things—a professional mentorship, a casual friendship, or a shared hobby. Jacob Eisenstein and Vinodh Krishnan Elangovan showed that by integrating conversational content with network topology, models can infer the latent "type" of a relationship that structure alone cannot clarify.
  2. Links Disambiguate Text: Words often have multiple meanings. However, if a document is embedded in a specific citation cluster, the network context can "nudge" the topic model toward the correct sense of a word. Ramesh Nallapati and Amr Ahmed demonstrated in their Joint latent topic models that treating citations as topical signals significantly improves the accuracy of text prediction.

From Probabilistic Models to Graph Neural Networks

The field has evolved from purely generative probabilistic frameworks to hybrid neural architectures that can handle larger, more complex graphs.

Jimin Huang and Pan Du 0002 introduced the Graph Relational Topic Model (GRTM), which uses Higher-order Graph Attention Networks (HGAT) to capture "friend-of-a-friend" relationships. This allows the model to learn from multi-hop connections without the "over-smoothing" problem common in traditional Graph Neural Networks. Similarly, the Layer-Assisted Neural Topic Model (LANTM) by Ximing Li integrates links as primary input features in a dual-channel encoder, ensuring that the latent space is informed by both text and graph structure at every layer of the network.

Modeling Evolution and Authority

Beyond static snapshots, this joint approach is particularly effective for tracking how ideas move through time and who the "gatekeepers" of those ideas are.

  • Temporal Dynamics: Ce Zhang 0004 and Hady W. Lauw developed NetDTM, which uses optimal transport to track how topics evolve alongside growing citation networks. This allows researchers to map the "innovation path" of a technology as it emerges and splits into new sub-fields.
  • Topical Authority: Influence is rarely global. A researcher might be an authority on "Quantum Computing" but have no influence in "Marine Biology." Models like Latent Topical-Authority Indexing (LTAI) by Dongwoo Kim and Alice Oh calculate citation probability as a function of both topic similarity and the author's specific expertise, providing a more nuanced measure of scholarly impact than simple citation counts.

By unifying the "what" (content) with the "who" and "how" (network structure), this research direction provides a more holistic framework for understanding the complex, interconnected nature of human knowledge and social interaction.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_joint_topic_modeling_and_network_represe_e4d35cd0,
  title = {Joint Topic Modeling and Network Representation Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-topic-modeling-and-network-representation-learning/rdr_e4d35cd051394a949389ba23503e313d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_joint_topic_modeling_and_network_represe_e4d35cd0_v1,
  title = {Joint Topic Modeling and Network Representation Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-topic-modeling-and-network-representation-learning/rdr_e4d35cd051394a949389ba23503e313d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}