Capping Reviewer Assignment by Who Vouched, Not by Account

AI conferences now assign papers to reviewers by algorithm, and attackers have learned to game the algorithm by creating fake reviewer accounts. A recent audit of OpenReview found 94 confirmed fraudulent reviewer and meta-reviewer profiles, and 92 of them passed university email checks by using address aliases (Identity Theft in AI Conference Peer Review). The best existing defense is randomized assignment. It caps the probability that any single reviewer account lands on any single paper, so that a colluder cannot be sure of reaching a target (Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments). That cap is per account. An attacker who controls ten accounts receives ten draws at the target paper, so the protection weakens in proportion to how many fake identities get through.

This note proposes fixing the unit of the cap. Each reviewer account would carry a signed, machine-checkable record of who admitted it to the pool. Admitters include an institution, a prior venue, or an area chair vouching within a fixed quota. The assignment algorithm then caps the total probability mass that all accounts sharing an admission chain can place on any one paper. A corrupt or careless admitter can inject only a bounded number of accounts, and those accounts share one probability budget. The attacker's worst-case chance of reaching a target paper therefore becomes a provable function of how many admitters they have compromised, and no longer depends on how many accounts they created. The pieces come from two literatures. Verifiable credentials (signed attribute claims that any third party can check against the issuer's public key) supply the provenance. The tiered structure of editorial boards (editors, area chairs, reviewers) supplies a natural tree of delegated vouching authority.

Core Claim

Sybil resistance in reviewer assignment should be enforced as a constraint inside the assignment optimization, keyed to verifiable admission provenance, rather than detected after the fact from behavior.

Let $x_{rp}$ be the marginal probability that reviewer account $r$ is assigned paper $p$. Let $s_{rp}$ be their similarity score. Let $\mathcal{G}$ be a partition of accounts into provenance groups, where a group is the set of accounts admitted under one leaf issuer, such as one area chair's vouching quota or one institution's credential. We solve

$$\max_x \sum_{r,p} s_{rp} x_{rp} \quad \text{s.t.} \quad \sum_{r \in g} x_{rp} \le Q_{gp} \;\; \forall g \in \mathcal{G}, p,$$

together with the usual load and coverage constraints and the per-account caps of the randomized assignment method. An attacker who controls the set of groups $S$ reaches paper $p$ with probability at most

$$\Pr[\text{attacker assigned to } p] \le \sum_{g \in S} Q_{gp},$$

by a union bound over the groups they control. This bound holds no matter how many accounts the attacker registers inside those groups or how they inflate their keyword similarity.

The new element relative to existing work is the group-level marginal cap whose groups come from credential chains. The existing partition constraints forbid two reviewers from the same institution from being co-assigned to one paper, for conflict-of-interest purposes (Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments). They do not bound the aggregate probability that a group reaches a paper. They also take affiliations as trusted inputs, and the identity-theft audit shows those inputs can be forged.

Research Program

The work has three layers, and the assignment layer is the core contribution.

Credential layer. We reuse standard machinery rather than invent cryptography. Decentralized identifiers resolve to public keys without a central registry. Verifiable credentials follow the issuer–holder–verifier pattern. Revocation uses compressed bitstring status lists, as surveyed in A Survey on Decentralized Identifiers and Verifiable Credentials. Claim-based access control across organizations without a shared directory follows Geng et al..

A reviewer presents a credential issued by an institution's HR or registrar system. Issuance is tied to an employment record, not an email address, which defeats the alias trap. Alternatively, a reviewer presents a vouching credential from an area chair.

The reviewer also derives a venue-scoped pseudonym: a deterministic tag computed from their credential and the venue name. It lets the platform reject a second registration by the same person at the same venue without linking that person across venues. This is the zero-knowledge component sketched in A Novel Zero-Trust Identity Framework for Agentic AI. We implement it with an existing anonymous-credential library and measure only its cost.

Delegation layer. The editorial hierarchy maps onto issuance authority. The program chairs issue each area chair a delegation credential that permits at most $k$ vouches, expires at the review deadline, and is revocable on the shared status list. The design borrows the signed, scoped delegation tokens of Position: AI Agents Need Authenticated Delegation, with one change: we drop that design's reliance on a central OpenID provider and the large-language-model policy translation it flags as vulnerable to prompt injection. Quotas are integers in a signed token, not natural-language policy.

This addresses the principal–agent problem in tiered boards. The principal cannot observe an area chair's every choice, but it can bound the damage any one area chair can do. This is the same logic as gating ledger commits on verified authority in A Blockchain-Based Approach For Embodied Carbon Management, applied here to probabilities rather than commits.

Assignment layer. We extend the randomized linear program and its flow-based sampler to include the group constraints. Since the groups partition the accounts, the constraints should keep the structure the sampler needs, but we must verify this. We also study how to set $Q_{gp}$. Uniform caps penalize large honest groups, such as a big department, so we propose caps that scale with the number of distinct verified persons in a group. Vouched-only groups keep a fixed small budget. The central empirical question is how much similarity this costs on real data.

Baselines

The naive reference is the deterministic similarity-maximizing assignment. Run on the same ICLR 2018 matrix with the same injected attackers, it was gamed with a 100% success rate in Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments. It defines 100% of attainable similarity.

The strongest competitor is that paper's randomized assignment with per-account cap $Q_{rp}=0.5$. It reportedly retains 90.8% of the optimal deterministic similarity on ICLR 2018, and 99.1% with partition constraints. It is the strongest because it is the deployed-style defense that already bounds per-account manipulation. We run it with identical similarity matrices, loads, solver, and sampler, and give it the same tuning budget over $Q_{rp}$ that we give over $Q_{gp}$. We found no reported result on Lacuna for randomized assignment under multi-account sockpuppet injection, so we will measure it ourselves. Beating it means a lower worst-case attacker success probability at matched similarity retention, or higher similarity at matched attacker success, across the attacker budgets described below.

The second competitor is the cycle-free greedy heuristic of Combating Collusion Rings Is Hard but Possible. It reportedly keeps over 96% of optimal similarity while removing short review cycles, so it shows what structural constraints cost without identity information. The third is the best graph anomaly detector from On the Detection of Reviewer-Author Collusion Rings From Paper Bidding, which reached 31% Jaccard similarity with the true colluders. We run it as a filter that removes flagged accounts before assignment. The fourth is ORCID-style gating, which we simulate by requiring each account to link to a publication record.

The key ablation keeps the group-cap linear program unchanged but assigns accounts to groups at random. It must be matched in group sizes. This separates the value of the provenance from the value of merely aggregating caps. A second ablation removes vouching quotas, so that groups are unbounded in size, which isolates the delegation layer.

Why not just…?

Why not just require ORCID or publication-record linking? This is the most serious alternative. The audited fraud included impersonation of established scholars, whose publication records are real, and the audit notes that ORCID does not prevent duplicate profiles (Identity Theft in AI Conference Peer Review). Linking also excludes early-career reviewers with thin records, which is exactly the population that vouching admits. Our simulation includes an impersonation attacker who passes publication linking. If such attackers turn out to be rare, linking plus per-account randomization may suffice. The experiment reports the attacker mix at which provenance caps stop mattering.

Why not just detect sockpuppets from bids? Detection from bids alone reaches only 31% Jaccard similarity, and 54% of colluders obtained at least one target paper without triggering any detector (On the Detection of Reviewer-Author Collusion Rings From Paper Bidding). Low-rank detectors did no better than random on the human-collusion data of A Dataset on Malicious Paper Bidding in Peer Review. Detection is complementary, and we include detection followed by group caps as a combined condition.

Why not just lower the per-account cap? Lowering $Q_{rp}$ trades similarity uniformly across all honest reviewers, yet the attacker still gains linearly with the number of accounts. Our curves of similarity against attacker success will show whether a cap tight enough to neutralize a ten-account attacker costs more similarity than provenance caps do. If it does not, the proposal fails.

Why not just use arXiv-style peer vouching without caps? Vouching limits who enters the pool, but a single corrupt voucher can still flood it. Vouching with quotas but no group caps is exactly our second ablation. If quotas alone close most of the gap, the honest conclusion is that the delegation layer does the work and the linear program change is unnecessary.

Proposed Experiments

We use the ICLR 2018 similarity matrix of 911 papers and 2,435 reviewers. Provenance groups come from affiliation and co-authorship structure in S2ORC, and we generate synthetic area-chair vouching trees with quota $k$ from 2 to 10.

We inject attackers that control $m$ accounts, with $m$ from 1 to 20. Their similarity to target papers is inflated by keyword stuffing, which is modeled on the audit. Their bids follow the "cycle" and "overlap" strategies recorded in the CMU malicious-bidding dataset. Attackers come in three types: a single outsider who compromised one institution email, an outsider who compromised one area chair's vouching quota, and an impersonator of a real scholar.

The decisive check is a plot of empirical worst-case attacker success against $m$. Per-account randomization should rise with $m$, while provenance caps should plateau at $\sum_{g\in S} Q_{gp}$. We plot this alongside the fraction of optimal similarity retained, which must stay near the 90.8% randomized baseline to be acceptable. We also report runtime on ICLR 2018 against the under-one-minute figure of the randomized method, and the per-registration cost of credential presentation and pseudonym derivation.

The proposal is falsified in either of two cases: provenance caps lose more than a few points of similarity relative to per-account randomization at matched security, or the random-grouping ablation performs as well as the provenance grouping.