Introduction
As AI policy moves at a breakneck pace—evidenced by the EU AI Act and various US Executive Orders—a critical gap has emerged: the lack of a formal mechanism for building scientific consensus. While individual research papers proliferate and high-level political summits occur, there is no "North Star" for regulation grounded in unified technical agreement. This position paper argues that NeurIPS, as the field's premier conference, should adopt an IPCC-like model to catalyze consensus on critical policy bottlenecks such as model evaluation and compute thresholds.
The Consensus Gap in AI Policy
Evidence-based policy typically follows a three-step pipeline:
- Evidence Generation: Individual research papers (the core strength of NeurIPS).
- Evidence Synthesis: Reports summarizing findings (e.g., the International Scientific Report on AI).
- Consensus Formation: A process where the scientific community reaches a general agreement on the implications of the evidence.
Currently, the third step is missing. While governments hold "AI Summits," these are often high-level political events that lack the technical depth and scientific "cachet" of the research community. Without a unified scientific voice, policy risks being based on fragmented data or political intuition rather than rigorous, expert-backed agreement.
Proposed Framework: The IPCC Model for AI
To bridge this gap, the author proposes a structural framework for NeurIPS inspired by the Intergovernmental Panel on Climate Change (IPCC). The IPCC succeeded not by forcing unanimity, but by using "calibrated language" to describe levels of certainty. NeurIPS should do the same—honestly reflecting disagreement while highlighting areas of solid ground.
The proposed "pilots" for NeurIPS include:
- A Standing Working Group: A year-round body of experts to shepherd the consensus process and provide sustained leadership.
- A Dedicated Policy Track: A pilot track to promote scholarship specifically aimed at forging consensus, such as meta-analyses of conflicting evidence.
- Formalized Deliberation: Using debates and surveys during the conference to quantify the "latent level of agreement" within the community on controversial topics.
Key Insights: Why NeurIPS?
NeurIPS is uniquely positioned to lead this effort because it "Pareto-dominates" other venues. With over 16,000 attendees, it represents the multidisciplinary nature of the AI community and has a history of pioneering norms, such as the "Reproducibility Checklist." Conclusions reached through a peer-reviewed, transparent process at NeurIPS would carry more weight and legitimacy than those from industry-led or government-led groups.
A key insight of the paper is that consensus does not mean 100% agreement. Instead, it means identifying "non-obvious yet valuable common ground," such as agreeing on the procedures to reduce uncertainty even if current results are debated. By focusing on specific policy bottlenecks, NeurIPS can provide the technical primitives needed for robust regulation:
- Evaluation Selection: Determining which benchmarks actually prove a model is safe or "state-of-the-art."
- Threshold Design: Providing a scientific basis for regulatory thresholds, such as the $10^{26}$ FLOPs limit often cited in policy.
Challenges and Implementation
Transitioning to a consensus-building role is not without risks. The heavy presence of industry funding and researchers at NeurIPS could lead to conflicts of interest in policy design. This must be managed via strict oversight, similar to the conference's existing ethics review process. Furthermore, wading into policy may expose researchers to political attacks, particularly in polarized environments.
However, the author argues that these risks are manageable. Just as climate scientists overcame deep disagreements to build a powerful consensus, the AI community can use sustained effort to find common ground. The success of this initiative depends on NeurIPS leadership and the community choosing to adopt these "unorthodox" roles in future conference cycles, specifically targeting NeurIPS 2025.
Summary updated