The Impact Market to Save Conference Peer Review: Decoupling Dissemination and Credentialing

Karthikeyan SankaralingamarXiv 2025 · DOI · arXiv

Introduction

The current academic conference system is struggling under a dual-identity crisis. It attempts to serve simultaneously as a venue for dissemination—the sharing of all methodologically sound research—and credentialing—the provision of a scarce signal of prestige used for hiring and tenure. By coupling these goals into a single binary "Accept/Reject" decision, the system has devolved into a "reviewer roulette" that rejects nearly half of all sound research to maintain artificial scarcity.

This paper proposes the Impact Market (IM), a three-phase architectural shift that decouples publication from prestige. The IM accepts all rigorous work while utilizing a crowdsourced "futures market" and a three-year validation loop to identify high-impact papers and reward accurate reviewers.

The Problem: Artificial Scarcity and the Zero-Cost Attack

In the current protocol, reviewers can reject sound work for subjective reasons like "lack of significance" with virtually no personal consequence. This creates a "zero-cost attack" environment where the prestige signal has almost no correlation with the actual long-term impact of the research. Roughly 40–50% of submissions fall into an "Equivalence Class"—they are technically sound and novel, yet many are rejected simply to keep acceptance rates low. This lottery system stifles innovation and places an immense, often fruitless, burden on the community.

The Impact Market Framework

The proposed Impact Market replaces the gatekeeper model with a three-phase system designed to incentivize quality and long-term thinking.

Phase 1: Publication as Validation

In the first phase, the Program Committee (PC) acts as a validator rather than a gatekeeper. Any paper that is methodologically sound, rigorous, and contains non-trivial novelty is accepted for publication. This shifts the focus from subjective "impact" to objective "correctness," moving the acceptance rate from approximately 15% to 50%.

Phase 2: Reputation-Based Investment

To recreate the prestige signal, senior community members (investors) are given "reputational tokens" to invest in published papers they believe will have long-term impact. This creates a Net Invested Score (NIS). To prevent a mere popularity contest, investors must spend 40% of their tokens on a randomly assigned "Scrutiny Budget" and 60% on their own "Expertise Budget."

Phase 3: Long-Term Calibration

Three years post-publication, the system calculates a Multi-Vector Impact Score (MVIS) based on citations, artifact adoption (e.g., GitHub forks), and community use. This "ground truth" is used to update the Investor Rating (IR) of the reviewers. If an investor successfully identifies a "gem," their future influence grows; if they bet on "duds" or engage in collusion, their influence is slashed.

Data and Empirical Validation

The paper utilizes historical citation data from 10 major computer science conferences (including ISCA, ASPLOS, and NeurIPS) to establish the baseline for research impact. This data confirms a "long-tail" distribution where a small minority of papers account for the vast majority of citations.

Cumulative and log-scale citation distributions illustrating the extreme long-tail nature of research impact across papers.

As shown in the citation analysis, the top 20% of papers typically account for approximately 85% of total citations. The Impact Market is specifically designed to identify these high-impact contributions more effectively than the current binary review system.

Detailed citation statistics across major CS conferences, highlighting the concentration of impact in the top 10 and 20 papers.

To test the IM, the author developed an Agent-Based Simulation ("IM-Rebel") comparing the IM against the Current Protocol (CP). Under "Crisis" conditions (low-skill reviewers), the CP only identifies 34.2% of the best papers, while the Impact Market recovers 86.7% of those same papers by leveraging expert agency and conviction betting.

Key Insights

  • The Power of Agency: Allowing experts to self-select papers in their domain (Phase 2) increased the retrieval of high-impact papers from 28% to over 85%.
  • Wisdom of the Minority: In the IM, a single high-conviction bet from a high-rated expert can outweigh 50 low-skill investors, allowing "sleeper" hits to surface.
  • Staked Reputation: By turning reviewing into an investment with a three-year feedback loop, the IM imposes a high cost on incompetent reviewing and "turf wars," which are currently "free" behaviors.

Limitations and Future Work

While promising, the IM faces challenges regarding the integrity of the MVIS (e.g., citation cartels) and the "cold-start" problem for new researchers. Furthermore, moving from a pilot to a full-scale replacement of the conference system remains a significant sociological challenge. Future work will focus on refining collusion resistance and developing "New-Author Bonuses" to ensure equitable visibility for emerging scholars.

Summary updated

Abstract

Top-tier academic conferences are failing under the strain of two irreconcilable roles: (1) rapid dissemination of all sound research and (2) scarce credentialing for prestige and career advancement. This conflict has created a reviewer roulette and anonymous tribunal model - a zero-cost attack system - characterized by high-stakes subjectivity, turf wars, and the arbitrary rejection of sound research (the equivalence class problem). We propose the Impact Market (IM), a novel three-phase system that decouples publication from prestige. Phase 1 (Publication): All sound and rigorous papers are accepted via a PC review, solving the "equivalence class" problem. Phase 2 (Investment): An immediate, scarce prestige signal is created via a futures market. Senior community members invest tokens into published papers, creating a transparent, crowdsourced Net Invested Score (NIS). Phase 3 (Calibration): A 3-year lookback mechanism validates these investments against a manipulation-resistant Multi-Vector Impact Score (MVIS). This MVIS adjusts each investor's future influence (their Investor Rating), imposing a quantifiable cost on bad actors and rewarding accurate speculation. The IM model replaces a hidden, zero-cost attack system with a transparent, accountable, and data-driven market that aligns immediate credentialing with long-term, validated impact. Agent-based simulations demonstrate that while a passive market matches current protocols in low-skill environments, introducing investor agency and conviction betting increases the retrieval of high-impact papers from 28% to over 85% under identical conditions, confirming that incentivized self-selection is the mechanism required to scale peer review.
BibTeX
@misc{sankaralingam2025impact,
  author = {Karthikeyan Sankaralingam},
  title = {{The Impact Market to Save Conference Peer Review: Decoupling Dissemination and Credentialing}},
  year = {2025},
  eprint = {2512.14104},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.14104},
}
Figure