Abstract
Community detection in bipartite networks often ignores the rich content of the items being interacted with, while text-only clustering ignores the powerful signal of user behavior. We present the Content-Augmented Stochastic Blockmodel (CASB), a unified generative model that integrates bipartite graph interactions with a probabilistic model of item text. By leveraging both signals, CASB achieves superior performance in identifying specialized research sub-communities on platforms like arXiv and INFORMS, effectively solving the "cold-start" problem for new items and providing a robust regularizer against interaction noise.
Introduction
Identifying "communities" or "clusters" in data is a cornerstone of modern recommendation systems and information filtering. Traditionally, researchers employ Stochastic Blockmodels (SBMs) to find patterns in how users interact with items—for instance, which scientists read which papers. However, these models often treat items as "black boxes," ignoring the rich information contained within the items themselves, such as the text of a scientific abstract. Conversely, text-only clustering ignores user behavior, which can be a powerful signal of intent and expertise.
This paper addresses this gap by introducing CASB (Content-Augmented Stochastic Blockmodels). The primary motivation is the arXiv preprint server, where broad categories (like "Cosmology") are often too coarse for researchers who require highly specific daily feeds tailored to their sub-niche. By simultaneously analyzing who interacts with what and the actual content of those items, CASB improves community detection accuracy and interpretability.
The CASB Model
The CASB model treats the problem as a bipartite graph where users and items belong to latent (hidden) communities and clusters. The model consists of two primary components:
- The Interaction Component: This assumes the probability of a user $j$ finding item $i$ relevant depends entirely on their respective group memberships, following a standard bipartite SBM structure.
- The Content Component: This is the model's core innovation. Each item cluster is associated with a unique probability distribution over words (a Multinomial distribution). If an item belongs to a specific cluster, its text is assumed to be generated from that cluster's specific "vocabulary profile."
Because calculating the exact relationship between all these variables is mathematically intractable, the authors utilize Mean-Field Variational Inference. This approach allows for the efficient approximation of hidden cluster assignments by optimizing a lower bound on the model's likelihood.
Experimental Evaluation
The model was evaluated on three real-world datasets:
- arXiv astro-ph.CO (Cosmology): 7,819 papers and 621 frequent users.
- arXiv hep-th (Theoretical High Energy Physics): 6,677 papers and 1,449 frequent users.
- INFORMS Conference Data: Five years of conference proceedings where authors are treated as "users" and their co-presenters/papers form the interaction network.
Key Results and Insights
Solving the "Misplaced Paper" Mystery
A compelling application of CASB involved a historical quirk on arXiv where "Galactic" and "Nongalactic" astrophysics papers were often lumped into the same category due to conflicting naming conventions. CASB successfully separated these two distinct research communities in an unsupervised manner. In the task of separating "Cosmology" from "Galaxy" papers, CASB using a bag-of-words representation achieved the best performance, with only 884 misplaced papers, significantly beating benchmarks like KMeans on LDA (936) and Link-PLSA-LDA (985).
Content as a Regularizer
Interaction data can be noisy; users may click on items by accident or out of curiosity. By forcing the model to explain the content of the items, the text acts as a "sanity check" that prevents the model from creating clusters based on random interaction noise. Furthermore, because the model understands content, it can address the cold-start problem, assigning a brand-new paper (with zero views) to a cluster based solely on its text.
Qualitative Success
On the INFORMS dataset, CASB successfully identified distinct, recognizable research communities. As shown in the word clouds learned by the model, it clearly distinguished between "Optimization" and "Transportation Logistics" themes based on the vocabulary profiles of the clusters.
Conclusion
CASB demonstrates that augmenting structural interaction data with content information leads to more robust and interpretable community detection. While the current model focuses on binary feedback and requires vocabulary truncation for scalability, it provides a strong foundation for future work. Potential extensions include incorporating more nuanced feedback (like ratings or dwell time) and developing dynamic models to account for the evolution of research interests over time.
Summary updated