Award prediction with temporal citation network analysis

Zaihan Yang, Dawei Yin, Brian D. DavisonSIGIR 2011 · OpenReview

Abstract

Each year, many ACM Special Interest Group (SIG) communities recognize outstanding researchers through prestigious awards honoring their profound impact and research contributions. This work investigates an automated mechanism to assist in selecting future award winners. By approaching the task as a researcher expertise ranking problem, we propose a temporal probabilistic ranking model that integrates content relevance with citation network analysis. Experimental results using real-world citation data and historical awardee records indicate that the proposed approach effectively models award patterns across various scientific communities.

Introduction

Selecting award winners in scientific communities is traditionally a manual, peer-driven process. While metrics like citation counts and the H-index provide snapshots of impact, they are often static and fail to capture the dynamic trajectory of a researcher's career. A major gap in existing expertise-ranking models is the omission of temporal information; a researcher prolific decades ago possesses a different authority profile than a rising star with high current momentum. This paper addresses the challenge of balancing historical prestige with contemporary relevance to predict future laureates.

The Temporal Authority Propagation (TAP) Model

The core contribution of this research is the Temporal Authority Propagation (TAP) model. Building upon the foundations of PageRank, TAP modifies how authority is distributed between authors by incorporating four key temporal factors:

  1. CareerTime: The duration of a researcher's active career, where longevity typically correlates with higher authority.
  2. LastRestTime: The time elapsed since the researcher's last publication; longer gaps negatively impact current reputation scores.
  3. PubInterval: The average frequency of publication, serving as an indicator of active participation.
  4. Citation Influence Ratio (CIR): A decay function that weights citations based on timing. Papers that continue to be cited long after their initial release are granted higher prestige than those with only immediate, short-lived impact.

These factors are synthesized into an Individual Temporal Importance (ITI) score. This score, combined with co-authorship frequency and citation counts, determines the weights in a directed graph. To ensure field-specific relevance, the model employs a hybrid approach, combining network analysis with the Okapi BM25 information retrieval algorithm.

Experimental Results

The researchers evaluated the model using a dataset from the ACM Digital Library comprising over 170,000 authors and papers. The ground truth consisted of 118 historical award winners from 1990 to 2009 across eight ACM SIG communities.

Performance Against Baselines

The TAP model consistently outperformed traditional metrics, including H-index, total citation counts (NumCit), and standard PageRank (PR). As shown in the performance comparison, TAP achieved the highest scores across all primary evaluation metrics.

Comparison of TAP against traditional metrics across NumTop10, NumTop20, and MRR.

Specifically, the model successfully placed 47 out of 118 actual award winners (39.8%) within its Top 20 recommendations, representing a 17.5% improvement over the best non-temporal baseline (in-domain citation counts).

Community Variance

An intriguing finding of the study is that award patterns differ significantly by sub-field. The model demonstrated high predictive accuracy for communities like SIGKDD (Data Mining) and SIGIR (Information Retrieval), while facing greater challenges in fields like SIGCOMM (Communication).

Model performance variance across different ACM Special Interest Groups (SIGs).

This variance suggests that different research cultures value different types of authority. For instance, the model achieved a precision score of 0.71 for SIGKDD, indicating it is remarkably effective at narrowing down potential laureates in that specific domain.

Conclusion and Future Work

The TAP model demonstrates that time is the "missing link" in citation analysis. By introducing decay functions and activity metrics, the model better reflects how scientific communities perceive prestige and persistence. Future work may involve tuning temporal parameters specifically for different research cultures and incorporating deeper semantic analysis of citation contexts to further refine prediction accuracy.

Summary updated

Abstract

Each year many ACM SIG communities will recognize an outstanding researcher through an award in honor of his or her profound impact and numerous research contributions. This work is the first to investigate an automated mechanism to help in selecting future award winners. We approach the problem as a researchers' expertise ranking problem, and propose a temporal probabilistic ranking model which combines content with citation network analysis. Experimental results based on real-world citation data and historical awardees indicate that some kinds of SIG awards are well-modeled by this approach.
BibTeX
@inproceedings{yang2011award,
  author = {Zaihan Yang and Dawei Yin and Brian D. Davison},
  title = {{Award prediction with temporal citation network analysis}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2011},
  url = {https://openreview.net/forum?id=BJbfe8bd-S},
}
Figure