The Limits of Inference Scaling with Imperfect Verifiers

This direction is merged into another direction: inference-time compute scaling (test-time compute scaling), imperfect verification bottleneck. Its last version stays readable and citable here.

Version 1 of 119 papers35 conceptsfrozen 2026-10-04

For years, the primary way to make Large Language Models (LLMs) smarter was to scale up training—more parameters, more data, and more GPUs. Recently, a new frontier has emerged: inference-time scaling. Instead of just training a bigger model, we can let an existing model "think" longer by generating many candidate answers and picking the best one.

While this approach—often called "Best-of-N" sampling or resampling—can make a small model perform like a much larger one, it is not a "free lunch." A growing body of research shows that this scaling eventually hits a hard ceiling. The bottleneck isn't the model's creativity, but the fallibility of the "verifier" used to judge its work.

The Promise of the Large Language Monkey

The intuition behind inference scaling is simple: if a model has a non-zero chance of solving a problem, we can eventually find that solution if we sample enough times. Bradley Brown, Ronald Clark, and colleagues explored this in Large Language Monkeys, showing that "coverage"—the probability that at least one correct answer exists in a pool of samples—follows a predictable scaling law. With enough samples, even a modest model can produce a solution to a difficult problem that would normally require a frontier-class model like GPT-4o.

However, having a correct answer in the pool is only half the battle. You also need to be able to identify it.

The False Positive Ceiling

In a perfect world, we would have an "Oracle" verifier that never makes mistakes. In reality, we use imperfect verifiers: unit tests that might miss edge cases, reward models that can be "gamed," or other LLMs that can be fooled.

As we increase the number of samples ($N$), we aren't just increasing the chance of finding a correct answer; we are also increasing the chance of generating a "false positive"—an incorrect answer that happens to satisfy the verifier. Arvind Narayanan, Benedikt Stroebl, and Sayash Kapoor argue in Inference Scaling fLaws that for weaker models, the probability of generating these deceptive solutions grows faster than the probability of generating a truly correct one.

Eventually, the system hits a performance ceiling. Beyond this point, adding more compute (more samples) actually hurts performance because the verifier is more likely to pick a confidently wrong answer than a correct one. Chuhan Wu and Ruiming Tang describe this in Towards a Universal Scaling Law of LLM Training and Inference as a fundamental bottleneck: any non-zero error rate in the judge creates an upper bound on effective scaling.

The Geometry of Verification

To understand why this ceiling exists, we can look at the "discriminative power" of the verifier. Arpan Mukherjee, Deniz Gunduz, and colleagues framed this as an Optimal Transport problem in Test-time Verification via Optimal Transport. They found that the effectiveness of scaling is strictly bounded by the difference between the verifier's True Positive Rate and its False Positive Rate.

A more precise mathematical limit is proposed by Florian E. Dorner and Fanny Yang in ROC-n-reroll. They show that the ultimate performance ceiling is determined by the slope of the verifier's ROC curve at the origin. This represents the verifier's precision at its most stringent threshold. If the verifier cannot perfectly distinguish between the "best" wrong answers and the "worst" right answers, no amount of additional sampling will ever push the system to 100% accuracy.

Coherent Errors and Task Difficulty

The difficulty of verification isn't just about the verifier's size; it's about the "coherence" of the errors. Yefan Zhou and Yilun Zhou point out in Variation in Verification that stronger generators produce "coherent" errors—mistakes that are logically consistent and look plausible. These are much harder for a verifier to catch than the messy, obvious mistakes of a weaker model.

This creates a paradox: as generators get better, they become harder to verify. On extremely difficult tasks, even frontier-class models acting as verifiers can perform near random chance, causing inference scaling to plateau almost immediately.

Scaling the Judge

If the verifier is the bottleneck, the solution may be to scale the verification process itself. Shalev Lifshitz and Yilun Du propose Multi-Agent Verification, where instead of one judge, a "jury" of diverse LLM-based verifiers votes on the candidates. By scaling the number of verifiers ($m$) alongside the number of samples ($n$), they demonstrate that the performance ceiling can be pushed higher, allowing for more robust weak-to-strong generalization.

Ultimately, the research suggests that inference scaling is a powerful tool, but its success depends on maintaining a "verification-generation asymmetry." As Yiyang Jin and colleagues note in ReVeal, we must ensure that it remains easier to check a solution than to create one. When that gap closes, inference compute stops yielding intelligence and starts yielding noise.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_the_limits_of_inference_scaling_with_imp_e93e97ca,
  title = {The Limits of Inference Scaling with Imperfect Verifiers},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-limits-of-inference-scaling-with-imperfect-verifiers/rdr_e93e97caf2444f2e8757ed9be46bbd6b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_the_limits_of_inference_scaling_with_imp_e93e97ca_v1,
  title = {The Limits of Inference Scaling with Imperfect Verifiers},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-limits-of-inference-scaling-with-imperfect-verifiers/rdr_e93e97caf2444f2e8757ed9be46bbd6b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}