RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation

Sihong Wu, Yiling Ma, Yilun Zhao, Tiansheng Hu, Owen Jiang, Manasi Patwardhan, Arman Cohan

TL;DR

While large language models (LLMs) are increasingly used to draft peer reviews, their feedback is often too generic to help authors improve their work. RBTACT addresses this by using author rebuttals as a natural preference signal to train models via Direct Preference Optimization (DPO). By learning from reviewer comments that actually triggered concrete paper revisions, RBTACT generates significantly more actionable and specific review feedback than much larger proprietary models.

What Problem Does This Paper Address?

Peer reviewing is a cornerstone of scientific progress, but it is notoriously time-consuming. While researchers are turning to LLMs to help draft review reports, current AI-generated reviews suffer from a major flaw: they are superficial and generic. A comment like "improve experiments" or "clarify the text" might be fluent, but it lacks actionability—it doesn't give authors concrete, implementable steps to fix the issue.

The authors of this paper argue that the peer-review process already contains a built-in, underutilized source of human feedback: the author rebuttal. When a reviewer leaves a comment, the authors either make a concrete change, promise a future edit, or defend their paper as-is. By analyzing these interactions, we can determine exactly what kinds of comments successfully guide authors toward improving their manuscripts.

Methods

To train a model that generates actionable feedback, the authors introduced a new task: perspective-conditioned segment-level review feedback generation. Instead of asking an LLM to generate a massive, multi-page review all at once, the model is given a full paper and a specific perspective (e.g., Experiments, Writing, Theory) and tasked with producing a single, highly focused critique.

The training pipeline consists of two main stages:

  1. Supervised Fine-Tuning (SFT): The authors fine-tuned a base Llama-3.1-8B-Instruct model on perspective-conditioned review segments to establish a strong baseline (RBTACT-SFT).
  2. Direct Preference Optimization (DPO): To optimize for actionability, the authors constructed preference pairs from real review-rebuttal interactions. Comments that successfully compelled authors to revise their papers were treated as the "preferred" (chosen) outputs, while comments that were merely defended or deflected were treated as "rejected."

The preference ranking follows a strict hierarchy based on author uptake: $$\text{Concrete Revision Performed (CRP)} > \text{Specific Revision Plan (SRP)} > \text{Vague Commitment} > \text{Defend As-Is} > \text{Deflect/Reframe}$$

To handle the massive context lengths required for reading full scientific papers, the authors utilized memory optimizations including FlashAttention-2, DeepSpeed ZeRO-2, and 4-bit quantization during the DPO phase.

Datasets

The authors introduced RMR-75K (Review-Map-Rebuttal), a large-scale dataset containing 75,542 examples curated from ICLR 2024 open-review data.

Unlike previous datasets that look at reviews as a single block of text, RMR-75K breaks reviews down into atomic key-point segments and maps each segment directly to the corresponding span in the author's rebuttal. Each pair is annotated with one of seven perspective labels and one of five rebuttal impact categories.

The automated pipeline used to construct this dataset involves structural filtering, semantic LLM matching, and rigorous human verification to ensure high-quality alignment.

Overview of the RMR-75K dataset construction pipeline

Key Insights and Ideas

  • Rebuttals as an Implicit Reward Model: The core "aha!" moment of the paper is treating the rebuttal phase as a natural, large-scale human preference dataset. Instead of hiring experts to manually rate review quality, the authors let the downstream actions of the researchers act as the ground truth for what constitutes "actionable" advice.
  • Deconstruction Enables Optimization: Evaluating actionability on full-length reviews is incredibly noisy because a single review contains a mix of praise, minor formatting complaints, and major technical critiques. Breaking the text down into perspective-conditioned segments makes preference learning tractable.
  • Robustness Across Severity Levels: One might worry that optimizing for author revisions would bias the model toward minor, easy-to-fix issues (like typos). However, the authors found that over 50% of major technical critiques (e.g., regarding Theory or Novelty) still resulted in concrete revisions or specific plans, proving that the rebuttal signal is robust for high-stakes feedback.

Key Results

The authors evaluated RBTACT against strong baselines—including its own SFT variant, specialized review agents (MARG), and massive frontier models like GPT-5-chat and DeepSeek-V3.2—using a test set built from ICLR 2025 papers.

  • Human Evaluation: In blind tests scored by PhD-level text annotators, RBTACT achieved the highest Actionability score (3.46 out of 5) and Specificity score (4.08 out of 5), outperforming GPT-5-chat (3.38 and 4.04, respectively).
  • LLM-as-a-Judge Pairwise Win Rates: When an LLM judge compared model outputs head-to-head, RBTACT achieved the highest average win rate across all models, securing a 57.1% win rate directly against GPT-5-chat.
  • Preserving Groundedness: Crucially, optimizing for actionability did not cause the model to hallucinate or become irrelevant; RBTACT maintained parity with the safest baselines on groundedness and relevance metrics.

Limitations and Future Work

  • Short-Horizon Uptake: The dataset relies on what authors say or do during the brief rebuttal window. It does not track whether promised changes are actually maintained in the final camera-ready version of the paper.
  • Domain Bias: The data is pulled entirely from OpenReview, focusing heavily on the computer science and machine learning ecosystem (ICLR). How well this generalizes to double-blind journals in biology, medicine, or the humanities remains an open question.
  • Feasibility vs. Scientific Correctness: While RBTACT is excellent at providing clear, actionable instructions (e.g., "re-run baseline X with hyperparameter Y"), it cannot automatically verify if the suggested experiment is physically or computationally feasible for the authors. Integrating code or data artifact verification is a promising direction for future work.

Summary updated

Abstract

Large language models (LLMs) are increasingly used across the scientific workflow, including to draft peer-review reports. However, many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses. We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning. Rebuttals show which reviewer comments led to concrete revisions or specific plans, and which were only defended. Building on this insight, we leverage rebuttal as implicit supervision to directly optimize a feedback generator for actionability. To support this objective, we propose a new task called perspective-conditioned segment-level review feedback generation, in which the model is required to produce a single focused comment based on the complete paper and a specified perspective such as experiments and writing. We also build a large dataset named RMR-75K that maps review segments to the rebuttal segments that address them, with perspective labels and impact categories that order author uptake. We then train the Llama-3.1-8B-Instruct model with supervised fine-tuning on review segments followed by preference optimization using rebuttal derived pairs. Experiments with human experts and LLM-as-a-judge show consistent gains in actionability and specificity over strong baselines while maintaining grounding and relevance.
BibTeX
@inproceedings{wu2026rbtact,
  author = {Sihong Wu and Yiling Ma and Yilun Zhao and Tiansheng Hu and Owen Jiang and Manasi Patwardhan and Arman Cohan},
  title = {{RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1696},
}

Loading related content...

Figure