Position Paper: How Should We Responsibly Adopt LLMs in the Peer Review Process?

Juhwan Choi, JungMin Yun, Changhun Kim, YoungBin KimOpenReview 2025 · OpenReview

TL;DR

The AI research community is drowning in submissions, leading to review quality issues. This paper argues against using LLMs to write reviews, proposing instead that they should act as assistive tools for reviewers. Specifically, LLMs can help verify reproducibility, check citations, and flag ethical concerns, freeing up human experts for deeper analysis and maintaining the integrity of scientific evaluation.

What Problem Does This Paper Address?

The field of Artificial Intelligence is experiencing an unprecedented surge in research output, leading to a dramatic increase in paper submissions to major conferences. This exponential growth has created a significant strain on the peer review process. The authors identify two primary issues: an increased burden on individual reviewers and a shortage of experienced reviewers. This often results in a decline in review quality, posing challenges for effective decision-making. Furthermore, the rise of Large Language Models (LLMs) has introduced a new challenge: the proliferation of LLM-generated reviews, which, if used uncritically, can lack depth, nuance, and critical analysis, potentially homogenizing feedback and eroding the trustworthiness of peer review. The paper aims to propose a responsible and effective way to integrate LLMs into the peer review process without sacrificing human expertise.

Methods

The paper advocates for a shift from using LLMs for full review generation to employing them as assistive tools for specific, targeted tasks. Their proposed framework involves integrating LLMs into three key stages of the peer review process:

  1. Reproducibility Verification: LLM agents would be used to automatically assess the reproducibility of submitted research. This involves verifying submitted code for completeness and executability, attempting to re-implement core methodologies based on the paper, and then trying to reproduce the reported experimental results. The output is a detailed reproducibility report for human reviewers.
  2. Reference Verification: LLM agents, potentially using retrieval-augmented generation (RAG), would analyze cited references to ensure they genuinely support the claims made in the manuscript. This includes identifying directly supported claims, potentially unsupported claims, and detecting citation padding.
  3. Ethics Review Flagging: LLMs would screen submissions and supplementary materials against conference-specific ethical guidelines to identify potential issues such as improper disclosure of human data, sensitive information, or prompt injection attempts. This would generate an "ethical risk report" for human reviewers.

The authors also propose a technical measure to deter unauthorized LLM review generation: embedding an invisible message in conference paper templates that LLMs can detect.

Datasets

Not discussed in the paper. The paper focuses on the process of using LLMs and does not detail specific datasets used for training or evaluation of LLM agents for these tasks. It references existing LLM capabilities and frameworks like Paper2Code and AutoReproduce.

Key Insights and Ideas

  • LLMs as Assistants, Not Replacements: The core argument is that LLMs should augment, not automate, the peer review process. Their strength lies in handling tedious, systematic tasks, freeing up human reviewers for higher-level critical thinking.
  • Targeted LLM Applications: Instead of generic review generation, LLMs are most valuable for specific, verifiable tasks like reproducibility checks, citation validation, and ethical flagging. This preserves the unique human element of scientific evaluation.
  • The Reproducibility Crisis is a Prime LLM Target: LLMs' ability to process code and text makes them ideal for tackling the persistent reproducibility crisis in AI research, a task often too time-consuming for human reviewers.
  • Preventing LLM Misuse: The paper acknowledges the risks of LLM-generated reviews (homogenization, lack of accountability, manipulation) and proposes technical and policy measures to mitigate them.
  • Hybrid Approach is Key: The future of robust peer review lies in a combination of automated assistance and human expertise, ensuring both efficiency and integrity.

Key Results

The paper is a position paper and proposes a framework rather than presenting empirical results from a deployed system. However, it references existing work and hypothetical scenarios to support its claims:

  • Reproducibility Verification: The proposed workflow (Figure 2) outlines a multi-step process for LLM-assisted reproducibility checks, aiming to surface issues early and provide reviewers with actionable reports.
  • Reference Verification: A case study using Gemini-2.5-Pro (Figures 6, 7, 8) demonstrated the LLM's ability to correctly classify claims as "Directly Supported Claim," "Potentially Unsupported Claim," and "Citation Padding" when provided with the claim and the cited paper. This suggests LLMs can effectively identify issues with citations.
  • Ethics Review Flagging: The proposed pipeline (Figure 4) aims to automate the identification of ethical risks based on conference guidelines, making the process more consistent and scalable.

Limitations and Future Work

Limitations:

  • LLM Fallibility: The authors emphasize that LLM failures to reproduce results don't necessarily mean the original work is flawed; LLM limitations, reasoning gaps, or manuscript ambiguity can be factors. LLM reports should be seen as preliminary indicators, not definitive judgments.
  • Contextual Application: Reproducibility verification needs to be applied contextually, with exemptions for non-computational research, to avoid stifling creativity and diversity.
  • Human Judgment Remains Paramount: LLM assessments should not replace human critical analysis; they are supplementary evidence.
  • Cost and Infrastructure: Implementing these LLM-based tools will incur significant computational costs, requiring careful consideration of funding models (conference budgets, submission fees, industry partnerships).
  • Data Privacy and Security: Using LLMs requires careful attention to data privacy and the use of secure environments, ideally with open-source, locally runnable models.

Future Work:

  • Refinement of Tools and Practices: Continued development of automated tools and peer review practices is necessary for fair and rigorous evaluation across all scientific work.
  • Integration with Existing Platforms: The proposed framework should be integrated into existing review platforms like OpenReview.
  • Transparency in Cost Allocation: Clear communication about how costs are calculated and allocated is crucial for community acceptance.
  • Safeguards Against Over-Reliance: Implementing reviewer overrides, transparency about system limitations, and structured guidance are needed to prevent over-reliance on LLM outputs.

Summary updated

Abstract

This position paper presents a novel perspective on the utilization of Large Language Models (LLMs) in the artificial intelligence paper review process. We first critique the current tendency for LLMs to be primarily used for simple review text generation, arguing instead that this approach overlooks more meaningful applications of LLMs that preserve human expertise at the core of evaluation. Instead, we advocate for leveraging LLMs to support key aspects of the review process—specifically, verifying the reproducibility of experimental results, checking the correctness and relevance of citations, and assisting with ethics review flagging. For example, integrating tools based on LLM Agents for code generation from research papers has recently enabled automated assessment of the reproducibility of the paper, thereby improving the transparency and reliability of research. By reorienting LLM usage toward these targeted and assistive roles, we outline a pathway for more effective and responsible integration of LLMs into peer review, ultimately supporting both reviewer efficiency and the integrity of the scientific process.
BibTeX
@misc{choi2025position,
  author = {Juhwan Choi and JungMin Yun and Changhun Kim and YoungBin Kim},
  title = {{Position Paper: How Should We Responsibly Adopt LLMs in the Peer Review Process?}},
  year = {2025},
  url = {https://openreview.net/forum?id=KZ3NspcpLN},
  howpublished = {OpenReview},
}
Figure