Key Institutional Considerations for Designing Effective Independent Verification Organizations

Uma Kalkar, Rumman Chowdhury, Reva Schwartz, Gabriella Waters

TL;DR

As the AI industry moves from "grading its own homework" to independent oversight, this paper argues that current designs for Independent Verification Organizations (IVOs) are dangerously narrow. The authors propose a robust institutional framework that shifts the focus from just "catastrophic" frontier risks to "everyday" harms, ensuring financial independence through pooled funding and requiring evaluations to be conducted in the specific context of use.

What Problem Does This Paper Address?

The AI industry has long operated with few constraints, often treating safety as a hindrance to deployment. While there is a growing bipartisan push for Independent Verification Organizations (IVOs)—expert-led bodies that impartially assess AI systems—the "momentum is outpacing design."

Currently, IVO discourse is fixated on "frontier" models (like GPT-4 or Claude) and "existential" risks (e.g., AI-enabled biological weapons). The authors argue this is myopic. It ignores the "enterprise" market—the thousands of companies deploying AI in specific sectors like healthcare or education—where the risks are "everyday" but no less impactful (e.g., bias in hiring or system unreliability). Without a clear institutional architecture, IVOs risk "regulatory capture," where the companies being evaluated dictate the terms, funding, and even the findings of the assessment.

Methods

This is a policy and institutional design paper rather than an experimental study. The authors employ a comparative institutional analysis, drawing lessons from established oversight regimes in other high-stakes fields:

  • Financial Services: Using the Public Company Accounting Oversight Board (PCAOB) as a model for separating who pays for an audit from who conducts it.
  • Public Health: Looking at the FDA’s "issuer-pays" model for drug trials.
  • Forensic Science: Adapting proficiency testing and "scientific foundation reviews" to ensure evaluation methods are actually valid.

The paper synthesizes these into a proposed IVO Ecosystem governed by four core principles:

  1. Findings are evidence, not verdicts.
  2. Those who set rules do not apply them.
  3. Those who apply rules do not police themselves.
  4. Those who pay for evaluations do not control them.

Datasets

As a policy-focused paper, it does not introduce new training datasets or benchmarks. However, it references:

  • Waters (2026) Survey: A survey of stakeholders regarding the utility and design of IVOs.
  • Stanford HAI 2026 AI Index: Data on AI adoption rates in North American business functions.
  • Legislative Review: A comprehensive mapping of current U.S. state (CA, CT, IL, VA, OH) and federal (FRONTIER AI Act) legislative efforts as of late 2026.

Key Insights and Ideas

  • The "Issuer-Pays" Trap: The authors warn against the current "voluntary accord" model where companies choose and pay their own evaluators. They cite the 2008 financial crisis—where banks "shopped" for AAA ratings from credit agencies—as a cautionary tale. Their solution? A mandatory pooled fund managed by an independent administrator.
  • Upstream vs. Downstream Information: Effective evaluation requires a "sandwich" of data. IVOs need upstream info from developers (training data, model weights) and downstream info from deployers (how the tool actually behaves when used by real people in specific workflows).
  • Evaluation in Context: A model is not "safe" in the abstract. The authors argue that an IVO finding for a specific LLM version used for drafting emails cannot be automatically transferred to that same model used for screening job applicants.
  • The Evaluator Monoculture: There is a significant talent overlap between frontier AI companies and the people qualified to evaluate them. The paper highlights the risk of an "ideological monoculture" and calls for formalized professional certification and "blind" proficiency testing to mitigate bias.

Key Results

The paper outlines a structured Tiered Reporting System to balance transparency with proprietary protection:

  • Full Findings: Shared with the company and the oversight body.
  • Case-Specific Findings: Shared with arbitration authorities for dispute resolution.
  • Anonymous, Aggregate Insights: Shared with standards bodies to improve the whole ecosystem.
  • Public Assurance Statement: A structured summary for the general public.

It also provides a roadmap for IVO Designation, suggesting that IVOs should not have "universal competence" but should be licensed for specific domains (e.g., an IVO certified for "AI in Education" might not be qualified for "Cybersecurity").

Limitations and Future Work

The authors acknowledge several "open problems" that require further research:

  • The Science of Evaluation: Much of the structure necessary to holistically design IVOs does not yet exist. We still lack validated methods for measuring many "everyday" sociotechnical harms.
  • Proprietary Friction: There is no established consensus on exactly how much access is "enough" to evaluate risk without compromising a company's trade secrets.
  • Ideological Conflicts: While financial conflicts are easy to manage (recusal), political or ideological biases in evaluators are much harder to identify and contain.
  • Method Obsolescence: With frontier labs releasing new models every few months, the paper notes the challenge of keeping IVO standards and evaluator training from becoming immediately outdated.

Summary updated

Abstract

Independent verification organizations (IVOs) have emerged as a popular, bipartisan solution for facilitating high-quality independent evaluations of AI technologies. But momentum is outpacing design: there is emerging consensus on the necessary operating conditions for robust IVOs, but little discourse on institutional and policy specifications to ensure those conditions are met. IVOs are currently envisioned primarily as a mechanism operating at the AI frontier and model level to evaluate catastrophic risks. Little work has yet been done on how to establish or enforce an IVO ecosystem that is equipped to handle myriad risks (i.e., beyond catastrophic ones) and operate at the system and application level across the deployment and use stages of the AI lifecycle. This paper argues that IVO design should ensure the capacity for evaluating AI models, systems, and applications in context for both frontier companies and enterprise companies & AI-adopting organizations. Additionally, this paper provides early considerations to advance IVO discourse from normative to operative, and from existential to everyday risks. Drawing from existing institutional constructs in financial services, education, public health, and science, we outline key considerations on whom IVOs serve, what access they require, how they interact with external governing structures, how they are funded, and how they fit into the wider assurance ecosystem. Much of the structure and knowledge necessary to holistically design IVOs and their ecosystem does not yet exist, even as companies rush to ink agreements with their chosen evaluators.
BibTeX
@misc{kalkar2026key,
  author = {Uma Kalkar and Rumman Chowdhury and Reva Schwartz and Gabriella Waters},
  title = {{Key Institutional Considerations for Designing Effective Independent Verification Organizations}},
  year = {2026},
  eprint = {2611.10168},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10168},
}
Figure