Abstract
In a document network such as a citation network of scientific documents, web-logs, etc., the content produced by authors exhibits their interest in certain topics. In addition some authors influence other authors' interests. In this work, we propose to model the influence of cited authors along with the interests of citing authors. Moreover, we hypothesize that apart from the citations present in documents, the context surrounding the citation mention provides extra topical information about the cited authors. However, associating terms in the context to the cited authors remains an open problem. We propose novel document generation schemes that incorporate the context while simultaneously modeling the interests of citing authors and influence of the cited authors. Our experiments show significant improvements over baseline models for various evaluation criteria such as link prediction between document and cited author, and quantitatively explaining unseen text.
Introduction
In academic publishing and social media, an author's interests are often visible through the topics they write about. However, quantifying their influence—how their work shapes the topical trajectory of others—is a more complex challenge. Traditional topic models like Latent Dirichlet Allocation (LDA) and its extension, the Author Topic Model (ATM), treat documents as isolated entities, failing to account for the rich network of citations and links that define influence.
Existing link-aware models face two primary hurdles: credit assignment and context noise. In multi-author papers, it is difficult to determine which specific author exerted influence. Furthermore, when a paper cites another, the entire citing document is rarely relevant to the reference; usually, only a small "context window" around the citation explains the connection. Using the whole document as context introduces significant topical noise.
Proposed Models: ALT and ACT
To address these limitations, the researchers introduced two novel generative models: the Author Link Topic (ALT) model and the Author Cite Topic (ACT) model. These models are designed to distinguish between an author’s personal research interests and their topical influence on the community.
- Author Link Topic (ALT): This model assumes that authors choose which peers to cite based on the specific topics they are currently writing about. It models influence as the probability that a cited author is referenced given a specific topic.
- Author Cite Topic (ACT): Building on ALT, the ACT model is context-sensitive. It operates on the assumption that the words immediately surrounding a citation and the cited author themselves are generated from the same underlying topic. This allows the model to isolate the specific reasons for a citation.
Dynamic Context Selection
A key innovation of this work is the move away from fixed-length context windows (e.g., 10 words before and after a citation). Fixed windows are often too short to capture a full explanation or too long, catching unrelated thoughts. The authors developed a Dynamic Context Selection algorithm that grows the context window sentence-by-sentence. The window stops expanding when the topical similarity between the context and the cited document begins to drop, ensuring a clean signal for modeling influence.
Experimental Results
The models were evaluated on two datasets: CiteSeer-DS1 (a curated set of 3,312 documents) and CiteSeer-DS2 (a larger, noisier set of over 18,000 documents).
Quantitative Performance
The Dynamic-ACT model achieved the lowest perplexity—a standard measure of how well a model predicts unseen text—compared to ATM, ALT, and fixed-window baselines. In cited author prediction tasks, ACT-based models significantly outperformed baselines in Precision@K, demonstrating a superior ability to recommend relevant citations based on text.
Qualitative Insights
The research successfully demonstrated that interest does not always equal influence. For example, an author might be highly active in a topic (high interest) but rarely cited for it (low influence). Conversely, pioneers who write seminal papers may have low volume but remain the primary "influence" for a topic for decades. Qualitative analysis of topics like "Bayesian Learning" identified famous pioneers as influential, aligning with academic reality.
Conclusion
By distinguishing between interest and influence and utilizing dynamic context selection, the ALT and ACT models provide a more nuanced understanding of document networks. While computationally intensive due to the reliance on Gibbs sampling, these models offer a powerful framework for tracking influence in scientific literature and potentially other networked environments like weblogs and social media.
Summary updated