VeriCUA: Verification as Agentic Exploration for Computer-Using Agents

Qiushi Sun, Da Yu, Alexander Knop, Amer Sinha, Lingpeng Kong, Ben Kao, Ravi Kumar, Pritish KamatharXiv 2026 · arXiv

In the rapidly evolving world of AI, we are moving from chatbots that talk to Computer-Using Agents (CUAs) that do. These agents navigate websites, manage files on desktops, and interact with mobile apps. But there’s a catch: how do we know if they actually finished the job?

A trajectory might look successful in a screenshot, but the file might not have actually saved, or the agent might have fulfilled only half of a complex instruction. Current verification methods—like asking a Vision-Language Model (VLM) to judge a few screenshots—often miss these subtle failures.

Enter VeriCUA, a new framework from researchers at Google and The University of Hong Kong that treats verification not as a single snapshot judgment, but as an active exploration task.

TL;DR

VeriCUA is a platform-agnostic framework that verifies computer-using agents by treating verification as an agentic exploration process. By dynamically gathering evidence through both visual (GUI) and programmatic (code-based) checks, it significantly reduces false positives and outperforms static judges across web, desktop, and mobile environments.

What Problem Does This Paper Address?

Verifying if an agent completed a task is surprisingly hard. Traditional "VLM judges" suffer from a partial view problem: they only see the screenshots provided to them. If the decisive evidence (like a specific database entry or a file deep in a folder) isn't in those frames, the judge guesses.

On the other hand, programmatic checkers (scripts that check for a file's existence) are often too rigid. They might look for a file at a specific path, but if the agent saved it elsewhere successfully, the script fails. Furthermore, building custom verifiers for every single app and platform is an engineering nightmare. We need a verifier that is as flexible and exploratory as the agents it's supposed to judge.

Methods: Verification as Exploration

VeriCUA flips the script by turning the verifier into an agent itself. Instead of a single pass, it follows a structured, four-stage "agentic" workflow:

  1. Context Inspection: Before judging, the verifier "scouts" the environment. It looks at available files, database schemas, and UI trees to understand the specific context of the application.
  2. Rubric Synthesis: Using the original instruction and the scouted context, it generates a fine-grained rubric of "hard" (mandatory) and "soft" (optional/hygiene) requirements.
  3. Active Exploration: The verifier then selects tools from a Hybrid Action Space to gather evidence. It might use GUI tools (like zooming in on a specific UI element or performing OCR) or Code tools (like querying an SQL database or checking file metadata).
  4. Rule-Based Arbitration: Finally, it combines these findings. Crucially, it distinguishes between asserted evidence (what the agent claimed it did) and attested evidence (what the environment actually proves happened).
The VeriCUA workflow: from scouting the environment to gathering hybrid evidence and reaching a final verdict.

This hybrid approach allows the verifier to see what a recording might have missed. For example, a screenshot might show a "Save Successful" message (asserted), but a code-based check can confirm the file's timestamp actually updated (attested).

VeriCUA's hybrid action space allows it to look at the trajectory, the surface UI, and the persisted state (files/databases).

Datasets

The researchers evaluated VeriCUA across a wide range of benchmarks to prove its platform-agnostic nature:

  • Online (Live Environments): AndroidWorld (mobile), OSWorld (desktop), and ScienceBoard (scientific desktop apps).
  • Offline (Recorded Trajectories): WebArena (web) and OSReward (a cross-platform human-labeled dataset).

Key Insights and Ideas

  • Scouting Prevents Hallucinations: By inspecting the environment before creating a rubric, VeriCUA avoids assuming file paths or UI behaviors that don't exist in that specific app.
  • The Power of "Abstention": Unlike standard VLM judges that must give a "Success" or "Fail" answer, VeriCUA can abstain if the evidence is inconclusive. This makes its "Decided Accuracy" much higher, which is vital for high-stakes data curation.
  • Hybrid Evidence is Non-Redundant: The study found that adding code-based state checks to GUI inspection increased the decision rate and recall without increasing false positives. Visuals corroborate the "what," while code confirms the "is it really there?"

Key Results

VeriCUA consistently outperformed both standard VLM judges and automatically generated scripts.

  • Higher Accuracy: In the AndroidWorld suite, VeriCUA achieved 90.5% accuracy, compared to 78.4% for a standard VLM judge.
  • Lower False Positives: In ScienceBoard, VeriCUA's false-positive rate was just 5.4%, whereas the VLM judge flagged nearly 25% of failures as successes.
  • Catching Liars: VeriCUA is particularly good at rejecting "false completion claims"—cases where the agent says "I'm done!" but hasn't actually finished. It rejected ~80% of these reports across online environments.
Performance comparison in online environments showing VeriCUA's superior accuracy and lower false positive rates.

Limitations and Future Work

While powerful, VeriCUA is not a silver bullet.

  • Cost: Because it performs multiple steps and tool calls, it is significantly more expensive (in terms of API tokens and time) than a single-pass judge. It is best suited for offline evaluation or reinforcement learning rather than real-time interactive use.
  • Read-Only Constraint: To ensure safety, VeriCUA never "writes" to the environment it verifies. While this prevents the verifier from breaking things, it also means it can't perform "destructive" tests (like trying to delete a file the agent was supposed to protect).

Future work may involve optimizing the cost of these agentic passes and expanding the toolset to even more complex enterprise software environments. For now, VeriCUA represents a major step toward building agents we can actually trust to get the job done.

Summary updated

Abstract

Building and scaling computer-using agents (CUAs) across desktop, mobile, and web environments requires reliable task-completion verification for data curation, evaluation, and reinforcement learning. Typical automated approaches prompt vision-language models (VLMs) to judge trajectories or generate programs to inspect application state. However, both rely on partial views of complex digital environments. VLM judges are restricted to supplied observations, while generated programs depend on accessible state. These views can omit decisive evidence or leave completion judgments unsupported. Tailoring separate verifiers to different platforms and access settings adds further engineering cost. We present VeriCUA, a platform-agnostic framework for verification through agentic exploration. It treats verification as dynamic evidence gathering and assessment within a shared procedure for online and offline settings across platforms and observation modalities. After inspecting the available context to construct a task rubric, the verifier selects GUI or code checks for unresolved requirements and combines their findings under explicit evidence rules. Experiments show improved agreement with reference labels over the compared baselines across these settings, with the largest offline gains on hard cases and stronger rejection of false completion claims online. Further analyses connect these gains to complementary evidence and test-time exploration.
BibTeX
@misc{sun2026vericua,
  author = {Qiushi Sun and Da Yu and Alexander Knop and Amer Sinha and Lingpeng Kong and Ben Kao and Ravi Kumar and Pritish Kamath},
  title = {{VeriCUA: Verification as Agentic Exploration for Computer-Using Agents}},
  year = {2026},
  eprint = {2610.40578},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.40578},
}
Figure