CollabRank: Towards a Collaborative Approach to Single-Document Keyphrase Extraction

Xiaojun Wan, Jianguo Xiao

Abstract

Traditional keyphrase extraction methods treat documents as isolated entities, relying solely on internal features like word frequency or local syntactic patterns. This paper introduces CollabRank, a framework that leverages "cluster context" by grouping similar documents and identifying salient keyphrases through a global word-relationship graph. By allowing documents to "collaborate," the system identifies core concepts that might appear moderately important in isolation but are central to a shared topic. Experimental results on news datasets demonstrate that CollabRank significantly improves extraction accuracy over single-document baselines and remains robust across various clustering methods and parameters.

Introduction

Most automatic keyphrase extraction methods operate under a "single-document" assumption, treating each text as an independent island. While effective for some tasks, this approach ignores the wealth of collaborative information available when processing sets of related documents, such as news articles covering the same event. A word that appears with moderate frequency in one article may be clearly central when observed across a cluster of related texts. The challenge addressed by the authors is how to effectively incorporate this cluster-level context to improve extraction for individual documents, particularly in genres like news where author-assigned keywords are often missing.

The CollabRank Framework

The authors propose a two-stage framework to leverage inter-document relationships:

  1. Document Clustering: The document set is first partitioned into topic-related clusters using standard algorithms such as K-means, Agglomerative (Average-link or Complete-link), or Divisive clustering.
  2. Collaborative Keyphrase Extraction:
    • Global Affinity Graph: For each cluster, a global graph is constructed where vertices represent words (filtered by part-of-speech) and edges represent co-occurrences within a sliding window across all documents in the cluster. A graph-ranking algorithm, similar to PageRank, is applied to assign a global saliency score to each word based on its importance within the entire topic.
  3. Document-level Scoring: Candidate phrases within a specific document are then scored by summing the cluster-level saliency scores of their constituent words.

An alternative method, RankFusion, was also explored, which combines these global cluster-level scores with local document-level statistics through a weighted linear combination.

Experimental Setup

To evaluate the system, the authors utilized 309 news articles from the DUC2001 dataset. Since no gold-standard keyphrases were originally provided, two graduate students manually labeled up to 10 keyphrases per document, creating a benchmark of 2,488 phrases. The original NIST-provided groupings (30 clusters) served as a "Gold Standard" to establish the upper performance bounds of the collaborative approach.

Key Results and Discussion

CollabRank consistently outperformed standard baselines, including SingleRank (the single-document graph-based equivalent) and TF-IDF. The method is remarkably robust to the choice of co-occurrence window size ($k$), maintaining stable performance from $k=2$ to $k=20$, as illustrated in the performance metrics below.

Precision, Recall, and F-measure of CollabRank across different clustering methods and window sizes compared to baselines.

Clustering Robustness

A critical finding is that while performance is highest with "Gold Standard" clusters (F-measure of 0.312), the system remains highly effective even with fully automatic clustering. Methods like K-means and Divisive clustering yielded F-measures between 0.302 and 0.310, significantly exceeding the SingleRank baseline of 0.272. However, the quality of clustering is paramount; using random clusters can degrade performance below the single-document baseline, as the "collaborative" signal becomes noise.

Impact of Keyphrase Count

The performance of the system also varies with the number of keyphrases ($n$) extracted per document. The F-measure typically peaks or stabilizes as $n$ increases, with CollabRank maintaining its advantage over the baseline across the entire range of extraction depths.

F-measure performance as a function of the number of extracted keyphrases (n).

Conclusion

CollabRank demonstrates that "the power of the crowd" applies to document analysis. By judging a word's importance based on its relationships across a topical cluster rather than just its local neighborhood, the system achieves more accurate and salient keyphrase extraction. Future work may involve exploring this collaborative framework in other domains beyond news and testing different underlying extraction algorithms within the same multi-document context.

Summary updated

BibTeX
@inproceedings{wan2008collabrank,
  author = {Xiaojun Wan and Jianguo Xiao},
  title = {{CollabRank: Towards a Collaborative Approach to Single-Document Keyphrase Extraction}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2008},
  url = {https://openreview.net/forum?id=NPmWTMgSRpO},
}
Figure