Joint Modeling of Latent Topics and Network Topology

39 concepts37 papers1 proposal

Documents rarely exist in a vacuum. Whether they are academic papers embedded in a citation graph, web pages connected by hyperlinks, or social media posts linked through user interactions, text is almost always situated within a network.

Traditional text analysis models, like Latent Dirichlet Allocation (LDA), treat documents as isolated bags of words. Conversely, traditional network analysis models focus purely on graph topology, ignoring the rich semantic content flowing through the edges.

The core insight of joint topic-link modeling is that content and connectivity are fundamentally intertwined. As Shuang Yang et al. point out in Mixture of Mutually Exciting Processes for Viral Diffusion, knowing the network structure helps identify the topic, and vice versa. By modeling both simultaneously, the network topology acts as a semantic anchor that guides topic discovery, while the latent topics provide the explanatory power behind why links form in the first place.

To understand why this approach works, we must look at how these models bridge the gap between discrete words and structural edges.

The foundational framework for this integration is the Relational Topic Model (RTM), introduced by Jonathan Chang et al. in Relational Topic Models for Document Networks. Instead of treating link prediction as a separate post-processing step, RTM integrates connectivity directly into the generative process of the text. It achieves this by conditioning the probability of a link between two documents on their latent topic assignments.

In a subsequent refinement, Jonathan Chang demonstrated in Hierarchical relational models for document networks that tying connectivity directly to the word-level generative process allows the model to learn topics that are simultaneously semantically coherent and predictive of relational links.

Traditional Approach (Sequential):
Text Content ───> Latent Topics 
                                ───> (Separate Link Prediction)
Network Graph ──> Graph Topology

Joint Approach (Interlocked):
Text Content  ───┐
                 ├───> Shared Latent Space ───> Coherent Topics & Link Prediction
Network Graph ───┘

Other mechanisms create even tighter local constraints to align text and topology:

  • Local Context Windows: Saurabh Kataria et al. developed a mechanism in Utilizing Context in Generative Bayesian Models for Linked Corpus that ties the latent topic of a citation or hyperlink to the specific words immediately surrounding it. Forcing the link and its local context to share the same topic assignment creates a semantic bottleneck that aligns the text and graph spaces.
  • Source-Target Matching: In Latent Topic Models for Hypertext, Amit Gruber et al. enforced a topic match between a specific source word and the overall topic distribution of a target document, treating links as explicit semantic bridges rather than independent tokens.

The Power of Mutual Regularization

Why does this joint optimization outperform decoupled, two-step approaches? The answer lies in mutual regularization: the text cleans up the network data, and the network data disambiguates the text.

1. Network Structure Disambiguates Text

Natural language is full of polysemy (words with multiple meanings) and noise. However, documents that share links are highly likely to share contexts. As Jonathan Chang notes in Hierarchical relational models for document networks, integrating network structure allows for the disambiguation of terms that share vocabulary but belong to different communities. For example, if two distinct research fields use the same jargon, they can be cleanly separated if their respective citation networks do not overlap.

Similarly, Zhen Guo et al. show in A latent topic model for linked documents that forcing the model to consider the topics of cited documents allows the corpus's connectivity to serve as a regularizer, preventing the model from being led astray by noisy or infrequent words in individual documents.

Purely structural link prediction models rely on topological features like degree distribution or common neighbors (e.g., "rich-get-richer" dynamics). By introducing text, the model gains an understanding of why a link should exist.

Duy Quang Vu et al. demonstrated in Dynamic Egocentric Models for Citation Networks that combining latent topic modeling with structural network statistics accounts for both thematic relevance and topological properties, leading to far more accurate predictions of link formation.

Dynamics, Roles, and Deep Learning

The principle of joint topic-link modeling easily extends to capture more complex network phenomena, such as time, user roles, and non-linear representations.

Temporal Evolution

Networks and text change over time. In Dynamic Topic Models for Temporal Document Networks, Ce Zhang 0004 et al. utilize network links as semantic anchors to align a document's latent topics with the word distributions of its neighbors across time steps. This temporal alignment ensures that the latent space remains grounded even as the network grows.

When tracking information cascades, Xinran He et al. show in HawkesTopic: A Joint Model for Network Inference and Topic Modeling from Text-Based Cascades that simultaneous inference creates a synergistic loop: accurate network mapping refines topic discovery, while topical consistency constrains the inferred influence graph.

Structural Roles

In social networks, links are often driven by specific user behaviors. Roy Ka-Wei Lee et al. integrated topic modeling with link analysis in Discovering Hidden Topical Hubs and Authorities in Online Social Networks to simultaneously learn latent user interests and their structural roles as topical influencers (authorities) or curators (hubs), ensuring that the discovered topics reflect social dynamics.

Deep Representation Learning

Modern iterations of this idea bridge probabilistic topic modeling with deep neural networks. Maosong Sun et al. inverted the metaphor in A Unified Framework for Community Detection and Network Representation Learning by mapping network nodes analogously to words, treating random walk sequences as text and network communities as latent topics.

To handle complex, multi-hop connections without falling into the over-smoothing trap common to deep graph networks, Jimin Huang et al. introduced the Graph Relational Topic Model (GRTM) in Graph Relational Topic Model with Higher-order Graph Attention Auto-encoders. By pairing a log-normal prior with a higher-order graph attention network, the authors demonstrated that aligning the latent space with both word distributions and multi-hop link structures yields significantly higher topic coherence than treating documents in isolation.

Key Foundational Work

Group and Topic Discovery from Relations and Their Attributes (2005) — contributed 1 concept elements to this direction.
Using Collocations for Topic Segmentation and Link Detection (2002) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_topic_aware_link_prediction_b99ee1ef,
  title = {topic-aware link prediction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/topic-aware-link-prediction/rdr_b99ee1ef54b84f3dba3562d7dcb270d5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_topic_aware_link_prediction_b99ee1ef_v1,
  title = {topic-aware link prediction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/topic-aware-link-prediction/rdr_b99ee1ef54b84f3dba3562d7dcb270d5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake