In the race to develop powerful artificial intelligence, speed is often at odds with safety. Developers face a "first-mover advantage" that incentivizes cutting corners on rigorous testing to reach the market or the research frontier first. This creates a classic collective action problem: while everyone might be better off if all players prioritized safety, the individual incentive to move fast often leads to a "race to the bottom."
Researchers are increasingly using Evolutionary Game Theory (EGT) and economic modeling to understand how to break this cycle without triggering "over-regulation"—a state where safety mandates become so burdensome that they stifle the very innovation they were meant to protect.
The Tradeoff Between Punishment and Reward
The most intuitive way to enforce safety is through punishment—fines, bans, or legal liability for unsafe releases. However, work by The Anh Han, Luı́s Moniz Pereira, and Francisco C. Santos in Mediating artificial intelligence developments through negative and positive incentives suggests that punishment is a blunt instrument.
While punitive measures effectively curb unsafe practices in high-stakes environments, they risk suppressing beneficial innovation in low-risk scenarios. If the cost of a mistake is a career-ending penalty, developers may avoid experimentation entirely. Their modeling shows that positive incentives—rewarding safe players—can act as a more flexible catalyst. Rewards encourage safety compliance without the risk of shutting down technological progress when the true risk level of a new technology is still unknown.
Escaping Over-Regulation Through Proportionality
To avoid the "one-size-fits-all" trap, researchers propose a framework of proportionality. The idea is to scale the intensity of oversight to the potential impact of the technology.
In The science and practice of proportionality in AI risk evaluations, Carlos Mougan and colleagues describe this as a Pareto-frontier problem. Regulators must find the point where they maximize safety evaluations while minimizing the "compliance tax" on developers.
This risk-proportional approach is echoed in academic settings as well. Anders Søgaard and Daniel Hershcovich argue in A Two-Sided Discussion of Preregistration of NLP Research that scientific oversight, such as mandatory preregistration of hypotheses, should only be required for high-stakes research. By keeping administrative barriers low for low-risk exploration, the community maintains its creative engine while securing its most impactful outputs.
The Innovation Regulation Trilemma
The challenge of governance is often described as a "trilemma" between three competing goals:
- Rapid Technological Advancement: Maintaining the speed of innovation.
- Public Safety and Market Integrity: Preventing catastrophic failures or systemic bias.
- Geopolitical Competitiveness: Ensuring that domestic regulations don't drive talent and capital to less-regulated jurisdictions.
Jon Chun and Christian Schroeder de Witt highlight in Comparative Global AI Regulation: Policy Perspectives from the EU, China, and the US how different regions prioritize these corners. The US often favors speed and competitiveness, while the EU prioritizes safety through the AI Act.
However, Eren Kurshan and David R. Byrd argue in The Agentic Regulator that static laws cannot solve this trilemma. Because AI models learn and evolve, regulation must become "agentic"—using modular, real-time monitoring that matches the speed of the models themselves.
The Evidence Trap and Seeking New Data
A major hurdle in designing these incentives is the "evidence gap." Regulators often wait for empirical proof of harm before acting, but in the AI industry, the data required to prove harm is often held by the developers themselves.
Stephen Casper, David Krueger, and Dylan Hadfield-Menell warn in Pitfalls of Evidence-Based AI Policy that demanding high evidentiary standards can be a strategic delay tactic used by industry. If you can't regulate until you prove a model is dangerous, and you can't prove it's dangerous without access to the model, oversight is stalled indefinitely.
To counter this, Rishi Bommasani and Li Fei-Fei propose a shift toward evidence-seeking policy in Advancing Science- and Evidence-based AI Policy. Instead of waiting for a disaster to justify a rule, policies should mandate transparency and incident reporting to proactively build the evidence base needed for future proportional regulation.
Voluntary Commitments as a Middle Ground
When formal "hard law" is too slow or too risky, voluntary commitments can provide a temporary escape from the trilemma. Tom Lenaerts and colleagues demonstrate in Voluntary safety commitments provide an escape from over-regulation in AI development that these commitments allow developers to signal their safety-consciousness to regulators. This reduces the pressure for top-down, restrictive mandates, allowing the industry to grow while establishing norms that can eventually be codified into more precise, evidence-based laws.