Do Researchers Benefit Career-wise from Involvement in International Policy Guideline Development?

Yuta Tomokiyo, Keita Nishimoto, Kimitaka Asatani, Ichiro SakataarXiv 2025 · DOI · arXiv

TL;DR

Researchers are increasingly asked to help shape public policy, but does this time-consuming work actually benefit their academic careers? By analyzing contributors to the 2021 WHO global air quality guidelines, this study proves that policy engagement directly boosts both academic and policy citations. Furthermore, the process sparks new, highly impactful collaborations, providing hard empirical evidence that institutions should actively reward scientists' involvement in policymaking.

Introduction

Scientists are often called upon to translate their complex research into actionable public policy, such as climate change reports or public health guidelines. While surveys suggest researchers feel this policy engagement expands their networks, many worry that these demanding activities lack tangible academic rewards or institutional credit. This study provides empirical evidence showing how participating in international policymaking impacts a researcher's quantitative career metrics—including citation counts, collaboration networks, and the innovativeness of future publications.

Methodology

The authors used the 2021 WHO global air quality guidelines as a case study, identifying 101 researchers involved in the guideline's development and pairing them with a highly specific control group of 101 non-involved peers.

The three-step matching process used to identify a control group: 1) Specialization Vector generation using text embeddings, 2) Similarity Ranking, and 3) Final matching based on historical performance metrics.

To find precise matches, the researchers generated 1536-dimensional text embeddings of the researchers' pre-2016 publications to capture their exact scientific specializations. They then matched researchers based on these specialization vectors alongside historical performance metrics (academic citations, policy citations, and publication volume) using Mean Squared Error (MSE). Finally, they tracked both groups over time, analyzing changes in citation rates, co-authorship networks, and publication impact metrics like the Disruptive Index ($D$). Data was sourced from Scopus for academic records and Overton for policy citations.

Key Findings

Policy Work Pays Academic Dividends

Engaging in policy acts as a catalyst for academic visibility, leading to a delayed but significant surge in both academic and policy citations compared to peers who stayed strictly in the lab. While the treatment and control groups had identical citation trajectories up to 2015, the guideline researchers saw statistically significant increases in policy citations starting in 2020, and academic citations starting in 2022.

Cumulative academic citations for treatment (guideline researchers) vs. control groups, showing a significant divergence in the years following the policy intervention.

Forcing Functions for Fresh Collaborations

The guideline development process acts as a powerful, multi-year networking event. It forces top-tier researchers from diverse geographic and disciplinary backgrounds to work together, spawning new co-authorships. The co-authorship network among guideline researchers grew from 67 connected members (pre-2016) to 86 members during the guideline development (2016-2021).

Impact and Innovation

Papers born from these new guideline-induced collaborations scored highest in both citation counts and the Disruptive Index—a metric measuring how much a paper shifts the current scientific paradigm. Additionally, a massive 55.2% of the academic references in the WHO guideline were authored by the guideline researchers themselves, compared to a baseline self-citation rate of 16.1% in general air quality research.

Conclusion and Future Work

This research demonstrates that international policy engagement enhances a researcher's academic standing and fosters innovative collaborations. While the study is limited to a single case study in air quality, it suggests that academic institutions should recognize and reward policy involvement. Future work will expand this methodology to multiple policy domains and longer time horizons to see if these career benefits hold true across the broader scientific landscape.

Summary updated

Abstract

Researchers are no longer limited to producing knowledge; in today's complex world, they also address societal challenges by engaging in policymaking. Although involvement in policymaking has expanded, direct empirical evidence of its career benefits remains underexplored. Prior survey-based studies suggest potential advantages-such as broader professional networks and enhanced opportunities-yet raise concerns about insufficient institutional support. Here, we examine the 2021 WHO global air quality guideline-a science-based regulatory guideline-as a case study. To evaluate the impact of guideline development on research outcomes, we match guideline researchers with a control group of peers sharing similar research topics and prior performance. Our analysis reveals that guideline researchers attain higher future citation counts in both academic and policy domains. New collaborations formed during development yield publications with higher citation impact and the disruptive index. Moreover, about half the guideline's references are derived from guideline researchers' papers, highlighting their central role in shaping the evidence base. These results provide empirical support for the career benefits of policy engagement. Our findings indicate that engaging in international guideline development offers tangible career incentives for researchers, and that institutions can enhance research impact and promote innovative scientific progress by actively supporting their researchers' participation in such initiatives.
BibTeX
@misc{tomokiyo2025do,
  author = {Yuta Tomokiyo and Keita Nishimoto and Kimitaka Asatani and Ichiro Sakata},
  title = {{Do Researchers Benefit Career-wise from Involvement in International Policy Guideline Development?}},
  year = {2025},
  eprint = {2503.22584},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.22584},
}
Figure