Abstract
Quantifying the impact of scientific papers objectively is crucial for research output assessment, which subsequently affects institution and country rankings, research funding allocations, academic recruitment and national/international scientific priorities. While most of the assessment schemes based on publication citations may potentially be manipulated through negative citations, in this study, we explore Conflict of Interest (COI) relationships and discover negative citations and subsequently weaken the associated citation strength. PANDORA (Positive And Negative COI- Distinguished Objective Rank Algorithm) has been developed, which captures the positive and negative COI, together with the positive and negative suspected COI relationships. In order to alleviate the influence caused by negative COI relationship, collaboration times, collaboration time span, citation times and citation time span are employed to determine the citing strength; while for positive COI relationship, we regard it as normal citation relationship. Furthermore, we calculate the impact of scholarly papers by PageRank and HITS algorithms, based on a credit allocation algorithm which is utilized to assess the impact of institutions fairly and objectively. Experiments are conducted on the publication dataset from American Physical Society (APS) dataset, and the results demonstrate that our method significantly outperforms the current solutions in Recommendation Intensity of list R at top-K and Spearman's rank correlation coefficient at top-K.
Introduction
In the academic "publish or perish" environment, citation counts serve as the primary metric for funding, hiring, and institutional rankings. However, this system is susceptible to manipulation. Researchers may engage in excessive self-citation or form "citation circles" with frequent collaborators to artificially inflate their perceived impact. While some self-citations are legitimate components of a research narrative, others are purely promotional. Standard metrics like the h-index or PageRank-based rankings often fail to distinguish between an independent peer citation and one stemming from a close professional relationship. The core challenge lies in identifying which Conflict of Interest (COI) citations are "positive" (scientifically necessary) and which are "negative" (potentially biased).
Methodology
The authors propose the PANDORA (Positive And Negative COI-Distinguished Objective Rank Algorithm). The workflow is built upon three main pillars:
COI Identification & Classification
The algorithm evaluates the relationship between citing and cited authors, categorizing them into four distinct types based on collaboration history, institutional affiliation, and co-citation patterns:
- Positive COI: Authors have collaborated previously, and their papers are frequently co-cited by independent third parties, indicating legitimate shared relevance.
- Negative COI: Authors have collaborated, but no independent papers cite them together, suggesting an insular citation pattern.
- Suspected COI (Positive/Negative): Authors share an institution but have not collaborated. The distinction again depends on whether independent third parties co-cite their work.
Dynamic Weighting and Integrated Ranking
Instead of treating every citation equally, PANDORA applies an exponential decay formula to "Negative" COI citations. The weight is reduced based on collaboration frequency, time spans, and citation history. "Positive" COIs are treated as standard citations. PANDORA then combines a weighted PageRank for paper impact with the HITS algorithm to evaluate the authority of authors and journals. A credit allocation algorithm is used to distribute a paper's impact fairly among co-authors.
Dataset
The study utilized a massive dataset from the American Physical Society (APS), focusing on Physical Review C and Physical Review E. The dataset includes approximately 71,287 publications from 1970 to 2013, providing a comprehensive citation graph and detailed metadata.
Key Results and Insights
PANDORA's performance was evaluated against state-of-the-art methods like FutureRank and CAJTRank.
Superior Accuracy
PANDORA consistently outperformed its competitors. In "Top-K" paper rankings, PANDORA achieved Recommendation Intensity (RI) accuracy rates between 0.656 and 0.8, significantly higher than other models.
The Co-Citation Litmus Test
A key insight of the PANDORA approach is using "co-citation" as a proxy for legitimacy. If the broader scientific community cites two papers together, a citation between those authors is likely valid. Conversely, if only the authors themselves link their papers, it is flagged as a negative COI.
Geographic and Institutional Trends
The analysis identified the US, Germany, and Israel as leaders in research impact. It also noted that larger institutions naturally have higher COI citation counts due to their extensive internal research infrastructures. PANDORA helps prevent this volume from overshadowing high-quality work from smaller labs.
Conclusion
While PANDORA offers a more nuanced approach to scientific evaluation, it has limitations. The current study focused on the physics community; citation behaviors in other fields may require model recalibration. Additionally, the algorithm relies on visible ties like co-authorship and affiliation, potentially missing "hidden" conflicts of interest such as personal friendships or mentor-student relationships where the mentor is not a co-author. Future work will focus on robust entity resolution and expanding the model to other academic disciplines.
Summary updated