IntelliAsk: Learning to Ask Critical Questions with Human-Aligned Rewards

Karun Sharma, Vidushee Vats, Shengzhi Li, Yuxiang Wang, Zhongtian Sun, Prayag TiwariOpenReview 2026 · Submission to ICLR 2026 · Withdrawn · OpenReview

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

Peer review relies on substantive, evidence-based questions, but existing LLM-based approaches often generate surface-level queries. We find that LLM-generated questions take over 50\% of their question tokens from a paper’s first page, while human reviewers draw on the full text. Human questions are also more insightful, showing effort and grounding, whereas LLM questions mostly reflect surface style. To address this, we extract 151k candidate questions from ICLR 2024 reviews and filter them through a multi-stage filtering process into Probe-15K, a set of 15.5k high-quality questions. From this, we create ProbeVote-500, where human annotators score questions along effort, evidence, and grounding. Using these labels, we train IntelliReward, a reward model built from a frozen Autoregressive LLM with trainable multi-head transformers over the final 50 token states. This architecture outperforms API-based SFT finetuning (Gemini 2.5 Flash, GPT-4.1) as baselines for reward. Applying DAPO with IntelliReward, we train IntelliAsk, a question-generation model aligned with human preferences and substantially stronger than existing fine-tuned review models. Finally, by releasing Probe-15K, ProbeVote-500, and IntelliReward, we provide an automatic evaluation benchmark for reviewer questions that measures groundedness, effort, and evidence.
BibTeX
@misc{sharma2026intelliask,
  author = {Karun Sharma and Vidushee Vats and Shengzhi LI and Yuxiang Wang and Zhongtian Sun and Prayag Tiwari},
  title = {{IntelliAsk: Learning to Ask Critical Questions with Human-Aligned Rewards}},
  year = {2026},
  url = {https://openreview.net/forum?id=2UVYeupOsM},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}
Figure