Relational Document Modeling for Interconnected Text Networks

This direction is merged into another direction: document independence assumption. Its last version stays readable and citable here.

Version 1 of 165 papers69 conceptsfrozen 2026-10-04

Traditional Natural Language Processing often treats documents as "isolated islands"—independent units of text that exist in a vacuum. Whether it’s a bag-of-words model or a modern Transformer, the standard assumption is that the meaning of a document is contained entirely within its own characters.

However, real-world documents are rarely isolated. They exist within a "connective tissue" of hyperlinks, citations, and social interactions. A research paper is defined as much by the papers it cites as by the words in its abstract. A web page’s identity is shaped by the anchor text of the links pointing to it. Research across several decades has shown that ignoring this structural context leads to a significant loss of semantic depth.

The Homophily Principle

The core intuition behind relational modeling is homophily: the idea that "birds of a feather flock together." In a network of documents, linked entities tend to share similar themes. Junyu Xuan, Jie Lu, and Guangquan Zhang argue that failing to account for this principle results in models that cannot accurately reflect the dependencies between content and connectivity.

When Jonathan Chang and David M. Blei introduced the Relational Topic Model (RTM), they proposed that the probability of a link between two documents should be a function of their topical similarity. By tying topic distributions directly to link probabilities, they showed that you could predict citations for new papers based solely on their text, or suggest keywords for a document based on its position in a network.

Bridging Text and Topology

If text provides the "what," the network provides the "where." Integrating these two requires moving beyond treating links as mere metadata.

One approach is to use the network to "smooth" or "regularize" the text representations. For example, Natraj Raman, Manuela Veloso, and Sameena Shah developed a quintuplet loss function that forces a model to learn embeddings where a document is closer to its cited neighbors than to unrelated documents, even if the vocabulary is different. This bridges the "semantic gap" where keywords alone fail.

In the realm of topic modeling, the Citation-Topic (CT) model by Zhen Guo and colleagues treats a document's content as a mixture of its own original ideas and the topics of the documents it references. This acknowledges that a document's meaning is often a synthesis of its neighborhood.

From Graphs to Foundation Models

As the field has moved toward Large Language Models (LLMs), the challenge has shifted to integrating massive graph structures into neural architectures.

  1. Structural Encoding: Mufei Li and colleagues proposed Struc-Emb, which moves beyond simple averaging of neighbor features. Instead, it integrates structural information directly into the LLM's internal encoding process, allowing the model to "see" the network while it reads the text.
  2. Graph Retrieval: Ali Maatouk introduced LitFM, a foundation model specifically for citation graphs. It uses a "neighbor-aware" retriever to provide the LLM with the structural context of "who-cites-whom," significantly reducing hallucinations in scientific tasks.
  3. Agentic Discovery: Models like DocReLM by Gengchen Wei and others use LLMs as agents that actively follow citation trails, mimicking how a human researcher navigates a library.

Why It Works: The Power of Human Curation

The reason relational modeling consistently outperforms text-only approaches is that links are human-curated signals of relevance.

In legal research, as explored by Pawan Goyal in the LeSICiN framework, the relationship between a case and a statute is a formal legal dependency that no amount of keyword matching can fully replicate. Similarly, in web search, Dou Shen and Zheng Chen showed that summarizing a page using its hyperlinked context filters out "noisy" elements like ads, focusing the classifier on the semantic core defined by the web community.

By treating the document collection as a graph rather than a list, these models leverage the collective intelligence embedded in the network's structure, turning the "isolated island" of a document into a well-connected hub of information.

Papers in this version

Showing 60 of 65 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_relational_document_modeling_for_interco_6183a59e,
  title = {Relational Document Modeling for Interconnected Text Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/relational-document-modeling-for-interconnected-text-networks/rdr_6183a59e11bd48db8f8426e994856bb2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_relational_document_modeling_for_interco_6183a59e_v1,
  title = {Relational Document Modeling for Interconnected Text Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/relational-document-modeling-for-interconnected-text-networks/rdr_6183a59e11bd48db8f8426e994856bb2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}