Introduction
The evaluation of scientific journals has long been dominated by the Journal Impact Factor (IF). However, the IF has faced significant criticism for being a simplistic average that fails to account for the "prestige" of the citing source and is susceptible to manipulation through self-citations. In response, modern influence scores such as the Eigenfactor (EIFA) and the Prestige Scimago Journal Rank (PSJR) have emerged. These metrics utilize PageRank-style algorithms on citation networks, weighing citations based on the influence of the journal providing them.
While these network-based metrics are an improvement, they rely on a "teleportation" mechanism—a damping factor (typically fixed at $\alpha = 0.85$)—to ensure the algorithm converges. This paper provides a rigorous statistical foundation for these scores by reinterpreting the PageRank smoothing matrix as the posterior mean of a Dirichlet-multinomial model within an Empirical Bayes framework.
The Problem: Arbitrariness and Zero-Handling
Traditional PageRank-based metrics suffer from two primary theoretical gaps:
- Arbitrary Damping: The choice of $\alpha = 0.85$ is a heuristic carried over from web search, applied uniformly to every journal regardless of its citation volume or behavior.
- Zero-Handling: Citation matrices are sparse. Standard algorithms often fail to distinguish between a "sampling zero" (a journal that could have been cited but wasn't in this specific window) and a "structural zero" (a journal that cannot be cited, such as when self-citations are intentionally excluded to prevent bias).
The Empirical Bayes Framework
The authors propose a probabilistic model that replaces the global damping factor with a data-driven, journal-specific parameter. By modeling the citation process through a Dirichlet-multinomial framework, the "random jump" or teleportation in PageRank is revealed to be mathematically equivalent to a Bayesian prior.
Journal-Specific Damping
Instead of a fixed $\alpha$, the model derives a shrinkage coefficient $\alpha_i$ for each journal $i$. This factor is determined by the number of references a journal produces ($n_i$) and a concentration parameter ($K$). For journals with sparse data (dangling nodes), the model automatically relies more heavily on the prior (teleportation). For journals with high citation volumes, the model allows the data to dominate the score.
Distinguishing Zeros
By explicitly modifying the Dirichlet prior, the authors can handle structural zeros rigorously. This is particularly important for managing self-citations. In many metrics, self-citations are simply removed, but this can bias the remaining probability distribution. The Empirical Bayes approach allows for the exclusion of self-citations without distorting the underlying statistical model.
The EBEF Algorithm
The authors introduce the Empirical Bayes Eigen Factor (EBEF). To estimate the model's hyperparameters, they employ a Majorize-Minimize (MM) algorithm. Specifically, a fixed-point iteration is used to maximize the marginal likelihood of the citation data. This process yields a concentration parameter ($K$) that reflects the actual density and behavior of the specific citation network being analyzed, rather than relying on a pre-set constant.
Case Study: 47 Statistical Journals
The methodology was tested using a cross-citation matrix of 47 statistical journals (data from Varin et al., 2016), covering citations from 2010 for articles published between 2001 and 2010. The dataset included prestigious titles like JASA and Annals of Statistics, alongside software-centric journals like the Journal of Statistical Software (JSS).
Key Findings
- High Correlation: The EBEF scores showed a Spearman rank correlation of approximately 0.99 with the standard Eigenfactor, validating the Bayesian approach as a theoretically sound alternative that reaches similar conclusions to established empirical methods.
- Stability of the Elite: The "top quintet" of journals—JRSS-B, STSCI, AOS, JASA, and BKA—remained stable across different scoring methods, indicating that the highest-tier journals are robust to algorithmic variations.
- Self-Citation Nuance: The model highlighted the impact of self-citations in software journals. For instance, the Stata Journal (STATAJ) has a self-citation rate of 67%. The EBEF framework allows for a more nuanced adjustment of these outliers than traditional methods.
- Algorithm Efficiency: The fixed-point iteration proved highly efficient, reaching parameter estimation in 5–7 seconds, while the Levenberg-Marquardt algorithm provided useful asymptotic variance estimates.
Conclusion and Future Directions
This research demonstrates that the "smoothing" required for journal rankings is not just a computational necessity but a statistical one that can be optimized using Empirical Bayes. By moving from a fixed damping factor to a journal-specific shrinkage coefficient, the EBEF provides a more tailored and theoretically grounded ranking system.
Future work may move away from the radical exclusion of self-citations toward a "data-driven underweighting" approach. This would involve a tuning parameter to attenuate self-citations based on field-specific norms, ensuring that journals are neither unfairly penalized nor artificially inflated by their own citation practices. Additionally, expanding the model to handle cross-disciplinary networks remains a key area for further development.
Summary updated