Abstract
Network-based document clustering involves forming clusters of documents based on their significance and relationship strength. This approach can be used with various types of metadata that express the significance of the documents and the relationships among them. In this study, we defined a probabilistic network graph for fine-grained document clustering and developed a probabilistic generative model and calculation method. Furthermore, a novel neural-network-based network embedding learning method was devised that considers the significance of a document based on its rankings with external measures, such as the download counts of relevant files, and reflects the relationship strength between the documents. By considering the significance of a document, reputative documents of clusters can be centralized and shown as representative documents for tasks such as data analysis and data representation. During evaluation tests, the proposed ranking-based network-embedding method performs significantly better on various algorithms, such as the k-means algorithm and common word/phrase-based clustering methods, than the existing network embedding approaches.
Introduction
Traditional document clustering usually relies on two primary signals: textual content and internal link structures, such as citations. However, these signals are increasingly susceptible to manipulation. "Abuse" in the form of keyword stuffing or artificial link creation can deceive standard algorithms, causing low-quality or spam documents to appear as central, authoritative nodes.
Beyond data integrity, standard algorithms often struggle to identify the most representative document within a cluster. For instance, in a large dataset of mobile app descriptions, identifying the "gold standard" app that users actually value is difficult using internal data alone. This research argues that internal data is insufficient for robust clustering; instead, an external "ground truth" of significance is required to anchor clusters effectively.
Methodology: The Net2Vec Framework
The authors propose a comprehensive framework for keyword-based probabilistic network clustering, centered around a new embedding technique called Net2Vec. The overall workflow involves selecting keywords, initializing clusters, and then expanding and merging them based on learned document embeddings.
Net2Vec and Authority Ranking Loss
The core innovation is the Net2Vec embedding method. While standard embeddings like Word2Vec or Doc2Vec learn by predicting context words or document neighbors, Net2Vec introduces a specific "Authority Ranking Loss."
As shown in the architecture comparison, Net2Vec adds a penalty term to the objective function. If a neighboring node has a high external ranking (such as high download counts) but the model predicts it with low probability, the model is heavily penalized. This mechanism forces the embedding space to "centralize" reputable documents, making them the anchors of their respective clusters.
Authority Ranking Score (ARS)
The mathematical engine behind this loss function is the Authority Ranking Score (ARS). It combines the external rank (calculated as the reciprocal of the ranking position) with the cosine similarity between nodes. By integrating external metadata as an anchor, the system becomes resistant to keyword stuffing; an attacker cannot move a document to the center of a cluster without also achieving high external popularity.
Probabilistic Generative Model
Rather than employing "hard-clustering" where a document belongs to a single group, the framework uses a generative model. It calculates the probability of a document belonging to a cluster based on its proximity to the authoritative center. This approach naturally supports multi-labeling, allowing a document (like a mobile app) to belong to multiple relevant categories simultaneously.
Evaluation and Results
The researchers evaluated Net2Vec using a large-scale dataset of mobile app descriptions from the Google Play Store, using actual download rankings as the external authority signal.
Quantitative Performance
In k-means clustering benchmarks, Net2Vec outperformed traditional methods like TF-IDF, LDA, and standard Doc2Vec:
- Purity: Net2Vec achieved the highest average purity (0.58).
- Entropy: It maintained the lowest entropy (1.95), indicating more distinct and well-defined clusters.
- Multi-labeling: In F-measure tests (balancing precision and recall), Net2Vec reached 0.72, surpassing standard Network Embedding (0.66) and Retrofitted Doc2Vec (0.69).
Qualitative Accuracy
Qualitative analysis showed that Net2Vec produces more semantically coherent clusters. For example, when searching for apps similar to Netflix, Net2Vec successfully identified other streaming services like Amazon Prime and Hulu. In contrast, standard network embedding methods often included apps that were technically linked (e.g., by developer) but served entirely different purposes.
Conclusion
By incorporating external popularity metrics into the embedding process, Net2Vec creates clusters that are both more representative and more resistant to data manipulation. While the method currently depends on the availability of external ranking data and faces a "cold start" challenge for new documents, it represents a significant step forward in fine-grained document analysis. Future work will explore applying this methodology to news and social networks, where authority can be derived from social signals like shares or verification status.
Summary updated