How Many Pages Must an Editor Read? Probe Budgets for Multi-Round Peer Review

An editor deciding the fate of a manuscript does not read everything. They skim the reviews, look up the one rebuttal paragraph that matters, and check whether a particular reviewer's objection survived to the final round. Data-structure theory has a clean way to count exactly this kind of behaviour: the cell probe model. In this model, memory is split into fixed-size cells, the only cost is the number of cells read, and all thinking between reads is free. This proposal applies that accounting to editorial decisions. We treat a multi-round review dossier as memory divided into fixed-size text chunks, organised by revision round. We then ask how many chunks a reader, human or model, must probe to predict the editorial outcome, and whether the reader gains by choosing each next chunk based on what it has already read.

The question matters because current AI work on editorial judgment answers it by accident. FirstPass fine-tunes a 7B model to predict whether a Nature Communications paper needs a standard or an extended revision (FirstPass). It fits documents into context with a fixed rule: keep the first 55% of the paper and the last 45% of the dialogue. That is a non-adaptive reading strategy, where what gets read is fixed in advance and does not depend on content. Nobody knows whether it is near-optimal, wasteful, or quietly exploiting a shortcut. A probe-budgeted evaluation turns "what should be read?" into a measurable curve. It also gives the editorial-governance literature something it lacks: a quantitative, theory-backed cost for a single tier's oversight.

Core Claim

Our first claim is about the shape of the trade-off between accuracy and the number of chunks read. Information about the editorial outcome is concentrated in a small number of chunks, and where those chunks sit depends on the manuscript. Two consequences follow. An adaptive reader with a budget of a few dozen chunks should match full-context fine-tuning. A non-adaptive reader with the same budget, such as fixed truncation or a fixed positional pattern, should fall clearly short.

This is the editorial analogue of the separation Adapt or Die proves for dynamic data structures. There, non-adaptive query algorithms for bit-indexing pay polynomially, with $t_q = \Omega(n/w)$ or $t_u = \Omega(k/w)$, while adaptive ones escape that cost.

Our second claim borrows the chronogram intuition, which partitions history into time epochs and shows a query must read fresh information from each epoch (Larsen, dynamic range counting; Bhattacharya, Henzinger, Neumann). Applied here, an accurate reader must spend probes in every review round that contains a still-open concern, not just in the most recent one. We will test this directly by stratifying probes by round.

The theory supplies the lens and a toy-model separation. It does not supply asymptotic bounds about journals. Dossiers are a few thousand words long, and $\Omega(\lg n/\lg\lg n)$-type statements say nothing at that scale. We say this plainly rather than oversell it.

Research Program

Protocol. The first deliverable is a probe-accounting protocol for FIRSTPASS (FIRSTPASS dataset), which contains 3,668 multi-round dialogues across five sciences. Each dossier is segmented into cells of $w$ tokens (with $w \in \{64, 256\}$), and each cell is tagged with its round and its role: paper, review, rebuttal, or reviewer update.

A reader receives a budget of $k$ cells. It may read them one at a time, choosing each next cell after seeing the previous ones, and then outputs a STANDARD or EXTENDED prediction.

Following the cell probe convention, the main cost is $k$. Because Panigrahy et al. show that cell probe accounting can miss the real bottleneck when internal processing dominates, we also report total model compute: tokens processed, including the controller's own reasoning.

Leakage control. This protocol must close a leakage path that matters for any study of this kind. The EXTENDED label means three or more rounds. If later-round material is visible, one probe that counts rounds solves the task trivially. In the strict setting, we therefore restrict the readable cells to material available at the end of round one: the paper, the first reviews, and the first rebuttal. We report a separate "full dossier" setting only as a leakage diagnostic.

Adaptive reader. The second deliverable is an adaptive reader. A small controller model looks at short headers for every cell (round, role, first sentence) plus the full text of cells already read, and picks the next cell. A Qwen2.5-7B classifier then makes the prediction from the cells read. This is the same base model and the same response-only loss masking that FirstPass found necessary. Without masking, FirstPass accuracy fell to 62%.

The controller is trained with policy-gradient on the classifier's log-likelihood, under a fixed budget $k$. The classifier is trained jointly on the cell sets the controller produces. We sweep $k$ from 4 to 128 cells, which traces the accuracy-versus-probes curve.

Theory. The third deliverable is a small theorem in a stylised dossier model. There are $R$ rounds and $m$ concerns, each concern opened and possibly resolved across rounds. The outcome is whether any concern remains unresolved. This is a cousin of the dynamic marked-ancestor problem studied in Further Unifying the Landscape.

We aim to prove two things. First, by an encoding argument, any non-adaptive reader needs $\Omega(m)$ probes to reach constant advantage. Second, an adaptive reader that follows the reply thread of a concern needs only $O(R \lg m)$. The point of the theorem is to make an empirical prediction falsifiable: the adaptive-versus-non-adaptive gap should grow with the number of distinct concerns per dossier. We can approximate that number by counting enumerated reviewer points.

Baselines

Do-nothing baselines. The naive reference is the majority-class predictor. FIRSTPASS reports a standard-to-extended split of roughly 63–71% across disciplines, so majority accuracy sits in that band on each domain split. Beside it we run a leakage probe: a one-cell reader that sees only the round count, evaluated in the full-dossier setting. If this probe scores near-perfectly there, it confirms why the strict setting is necessary.

Strongest existing method. This is FirstPass itself: Qwen2.5-7B-Instruct with LoRA, response-only loss masking, and 55/45 symmetric truncation. It reports 80.5% accuracy and 78.2% macro-F1 on outcome prediction, 10.4 points above zero-shot Gemini-3.1-flash-lite-preview (FirstPass).

We will re-run it ourselves under the strict round-one input restriction, with the same splits, the same LoRA rank and training steps, and a tuning budget equal to that of our classifier. It is unclear from the file whether its 80.5% was computed on round-one-only inputs, so the re-run, not the published number, is the comparison point. We will also measure its probe count, which is the number of $w$-token cells its truncation retains.

Beating it means one of two things on held-out data, judged with paired bootstrap confidence intervals. Either we match its accuracy within one point using at most a quarter of its cells, or we exceed it at equal cells.

Surface-feature baseline. We run the PeerRead-style surface-feature classifier on FIRSTPASS and, where possible, on PeerRead's accept/reject task (PeerRead). The original reports 65.3% accuracy on ICLR and 75.7% on CS.CL with 22 features. These are effectively "zero-probe" global statistics, so they show how much of the outcome is predictable without reading content at all.

Ablations. Two ablations isolate the key ingredient, adaptivity. The first gives a non-adaptive reader the same controller architecture, but it must commit to all $k$ cell choices from the headers alone before reading any text. This holds selection capacity fixed and removes only adaptivity. The second is an epoch-forced reader that must spread probes evenly across rounds. It tests the chronogram claim against unconstrained adaptive allocation.

Why not just…?

Why not just use a long-context model and read everything? If full reading wins, the probe curve will show it. The curve's top point is the full-context reader. The cost of reading everything is precisely what we measure, and FirstPass already needed truncation to fit dossiers into context, so "read everything" is not free. What remains open is whether all that reading buys accuracy. If a 32-cell adaptive reader matches full context, the long-context model is spending most of its compute on chunks that do not matter.

Why not just use embedding retrieval to pick the top-$k$ chunks? This is the real threat, because retrieval is already an adaptive reader. It picks chunks by their similarity to a query. We include it as a baseline: top-$k$ cells by similarity to a fixed query asking which concerns remain unresolved. If it matches the learned controller, the contribution reduces to the protocol, the leakage finding, and the shape of the trade-off curve. We would report that honestly. The theory predicts where retrieval should fail. Similarity search cannot follow a reply thread from a concern to its rebuttal to the reviewer's update, so its gap to the learned controller should widen on dossiers with many concerns. We pre-register that stratified test.

Why not just try a few truncation rules, such as keeping only the reviews or only the rebuttal? We run several fixed positional rules at every budget. They are the cheapest non-adaptive competitors. If one fixed rule matches adaptive reading, the adaptivity claim is refuted for this task. That result would still be useful: it would give editors and model builders a principled replacement for the ad-hoc 55/45 split.

Why not just model editorial latency with queueing theory? Queueing models describe how long items wait between people. They say nothing about how much of a dossier a decision-maker must inspect in order to decide correctly. Our question is about information, not waiting time, so the two approaches do not compete.

Proposed Experiments

The main experiment plots accuracy and macro-F1 against probe budget $k$ on FIRSTPASS, in the strict round-one setting. The curve covers all readers: adaptive, non-adaptive-with-same-architecture, retrieval, fixed truncations, and FirstPass's own rule. We report it per discipline and with leave-one-discipline-out transfer, to check that the selection policy is not memorising domain layout.

The first decisive check is the gap between the adaptive and non-adaptive readers at $k \in \{8, 16, 32\}$, stratified by concerns per dossier. The theory predicts the gap grows with concern count. A flat or shrinking gap would falsify the mapping.

The second decisive check is the chronogram test. We examine where adaptive readers place their probes across roles within round one, and, in the full-dossier diagnostic setting, across rounds. We compare the epoch-forced reader against unconstrained adaptive allocation.

The third decisive check is the leakage audit. We contrast strict and full-dossier accuracy, and the one-cell round-count probe, to establish what fraction of reported editorial-prediction accuracy reflects reading the science rather than counting rounds. We regard this audit as useful even if every other hypothesis fails.

For transfer, we repeat the trade-off curve on PeerRead accept/reject, where reviews and decisions are available. This checks whether the shape generalises from a single journal with only accepted papers to conference data that includes rejections. That addresses the selection bias that both dataset papers acknowledge.