Attested Verifiers for Untrusted Agent Teams

Many agent pipelines that work well have the same shape. One model proposes something, such as a piece of code or an intermediate mathematical claim. A separate checker accepts or rejects it, and the proposer tries again. This proposal places the boundary of a hardware enclave exactly along that split. An enclave is a region of a processor whose memory the host machine's owner cannot read or alter, and which can produce a signed report proving what code ran inside it. The large generator stays outside on ordinary GPUs and is not trusted at all. Only the small checker, the secret test material it uses, and the bookkeeping of the loop run inside the enclave. The enclave then signs a certificate of the form "every step admitted into this answer passed this specific, measured check, and the loop ended according to its stated rules." That claim has real semantic content, because the check is a deterministic test run rather than a fallible safety classifier.

The setting that makes this matter has three parties who do not trust each other. An agent provider wants to keep its prompts and model private. An asset owner holds secret tests or a proprietary rulebook that must not leak to the provider. A consumer or auditor wants proof that the delivered result passed those tests. Today the options are to trust the provider, to reveal the tests, or to wrap the whole agent in an enclave and attest only a final filter. The new question this design opens up is quantitative: how much feedback can the enclave pass back to the untrusted generator before the loop loses the accuracy gains of generator–verifier decomposition, and before the generator can reconstruct the secret tests? We propose to measure that trade-off and to build a protocol that sits at a good point on it.

Core Claim

The claim is that splitting the generation role from the verification role is also the natural split between untrusted and trusted computation. Making the two splits coincide gives three things at once. The trusted part of the system shrinks to the verifier plus a small state machine. The certificate covers the process of the multi-agent loop rather than one post-hoc check. Confidential verification assets stay sealed while still guiding iterative repair. We expect that a feedback channel carrying a small, budgeted amount of information can keep most of the accuracy of a full Tester loop, while keeping the extra pass rate an adversarial generator can gain on the hidden tests small. The core claim is a measurement of how much of that accuracy survives coarse feedback.

Where This Sits

The closest prior system is Proof-of-Guardrail in AI Agents and What (Not) to Trust from It. It runs a private agent and a public guardrail together inside an AWS Nitro Enclave and attests that the guardrail ran on the returned response. The authors themselves stress that proof of execution is not proof of safety. Their guardrail, Llama Guard 3, reaches an F1 of only 0.56 on unsafe content, and the whole agent sits inside the trusted boundary as one unit. We differ in three ways. The generator sits outside the enclave and is untrusted. The verifier is a deterministic executor whose verdict means what it says. The attestation covers every round of an iterative loop rather than a single final filter. Their paper motivates our work, because it found the semantic gap. It also constrains it, because it reports latency rising from about 421 ms to 547 ms and enclave instances costing roughly 18.5 times more than small bot instances, so the enclave must do as little as possible.

On the multi-agent side, two papers supply the method we reuse. Self-collaboration Code Generation via ChatGPT splits coding into an Analyst, a Coder and a Tester. It reports a 29.9% relative Pass@1 gain on HumanEval and gains of up to 34.6% on the edge-case-heavy ET suites, which they attribute to the Tester. MACM uses Thinker, Judge and Executor roles to raise GPT-4 Turbo accuracy on Level 5 MATH from 54.68% to 76.73%. Both assume a single trusted operator who sees everything. In our setting the Tester's tests are secret from the Coder, so the loop has to work with narrowed feedback. Whether it still works is exactly what we measure.

Two lines of work bound the systems design. Automated TEE Adaptation With LLMs moves sensitive leaf functions into Rust enclaves and measures the cost of crossing the enclave boundary. That cost is 1.6x for heavy routines such as RSA key generation but 1255x for a light call such as Python SHA256. Separately, The Orchestrator Bottleneck finds that once coordination consumes more than about 15% of the token budget, the quality gains of decomposition stop paying for themselves. Together these say that verification has to be batched at the level of a whole candidate, never per test or per token.

Two further systems handle different threats. BlockA2A secures messages between agents with ledgers and signatures but leaves each agent's memory visible to its host. Marshaled Learning fits large networks into small enclaves by cutting them at tensor layers. We cut at a semantic role instead, which is what lets the large model leave the enclave entirely.

Research Program

We will build a minimal enclave service on a function-specific enclave rather than a full confidential virtual machine, so that the trusted code stays small. The service holds four things. The first is a sealed test suite or rule set, provisioned by the asset owner after remote attestation, which is the hardware-signed report of the measured code. The second is a sandboxed interpreter that runs candidate programs, or MACM-style condition checks written as Python, against that suite. The third is a small state machine that enforces the loop's rules: a maximum number of rounds, a feedback budget, and the rule that only candidates that pass can be released. The fourth is a hash chain over every candidate and verdict. When the loop ends, the enclave signs the chain head, the measurement of its own code, and a commitment to the test suite. Anyone holding the certificate can then check that the released artifact is the one that passed, under the stated rules.

The generator is a stock Analyst–Coder pipeline, or a Thinker for MATH, running outside the enclave. It sees only what the feedback policy allows. We will study feedback at five levels of detail:

  1. a single pass or fail bit per candidate;
  2. the number of failed tests;
  3. a category for the failure, such as exception, wrong output or timeout;
  4. one failing input, released only under a per-session budget;
  5. full tracebacks with inputs, which corresponds to the Self-collaboration Tester.

To quantify leakage, we split each benchmark's tests into a hidden half, sealed in the enclave and used for feedback, and a shadow half, never used for feedback and evaluated only once at the end. The leakage gap is

$$\Delta = P_{\text{hidden}} - P_{\text{shadow}},$$

where $P_{\text{hidden}}$ and $P_{\text{shadow}}$ are the pass rates of the released programs on the two halves. A generator that genuinely improves raises both rates. A generator that fits itself to the feedback raises only the first. We will measure $\Delta$ against an honest generator and against an adversarial generator that is prompted to submit probe programs, for example programs that branch on their input and print it, in order to extract tests. The output of the program is a curve of shadow-test accuracy against $\Delta$, traced out across feedback policies and budgets.

Two systems questions will be measured directly rather than assumed. The first is the boundary cost per round, amortized by sending a whole candidate and running all tests inside one call. The second is whether timing differences in verdicts leak which test failed. For the second we will pad verdict latency to a fixed bound and report what that costs. We treat microarchitectural attacks of the kind shown in SgxPectre Attacks as outside our threat model and say so explicitly.

Baselines

The naive reference point is a single coder model with no verification loop and no enclave. It is run with the same base model, the same prompt and the same sampling settings, and gives the floor for both accuracy and cost.

The strongest existing method on accuracy is the full Self-collaboration pipeline, run with the Tester seeing all tests and returning full feedback, entirely outside any enclave. Its reported numbers are a 90.2% Pass@1 on HumanEval with GPT-4 and the relative gains above, all from Self-collaboration Code Generation via ChatGPT. We will rerun it ourselves with our base model and identical round limits rather than compare against the published figure, because the base models will differ. For MATH Level 5, the corresponding reference is MACM's reported 76.73% for GPT-4 Turbo, rerun the same way. Beating these baselines does not mean exceeding their accuracy. It means retaining at least about 90% of their gain over the naive floor on shadow tests, while keeping $\Delta$ near the honest-generator level and keeping tests sealed, which neither baseline attempts.

The strongest existing method on verifiability is Proof-of-Guardrail adapted to code. The whole agent runs inside one enclave with the tests, and a single final pass or fail check is attested, with no loop. It gets the same model, the same tests and the same instance type. Our system beats it if it achieves higher shadow accuracy at equal or lower enclave memory and cost, with a smaller trusted code base. We measure the trusted code base as lines of code and the size of the measured image.

Two ablations isolate the key ingredient. The first runs our exact loop and feedback policy with the verifier outside the enclave. This isolates the pure cost of hardware isolation, which we expect to resemble the 25–38% latency overhead Proof-of-Guardrail reports. The second keeps the enclave but replaces the deterministic test executor with a small LLM judge. This tests whether the semantic strength of the certificate really comes from the executor; we expect accuracy and certificate meaning to degrade toward the guardrail regime.

Why not just…?

Why not just run the whole agent team inside one confidential VM? This is the most serious competitor. It would match full-feedback accuracy exactly, because nothing about the loop changes. It falls short on the question the proposal asks. It requires the provider to load its private model into hardware it may not control, and it forces generation onto enclave-capable hardware at the roughly 18.5x premium Proof-of-Guardrail reports. It also enlarges the trusted code base to a whole guest operating system, which is the concern raised in Marshaled Learning. Our comparison against the single-enclave baseline reports accuracy, cost and the size of the trusted code base side by side. If accuracy at coarse feedback collapses, this alternative wins and we will say so.

Why not just have the asset owner run the tests on their own machine? If the owner is online and does not need to convince a third party, this works and needs no enclave. It fails when the owner is not online for every round, when many rounds make round trips expensive, or when a separate auditor needs a portable proof. Without an enclave, nobody but the owner can attest that the delivered program passed the sealed tests. We will report the number of rounds and the latency per task, so readers can see when the owner-online setup is good enough.

Why not just attest a final check and let the provider retry privately? A provider that can retry against a pass or fail oracle is already running our loop with one-bit feedback, but with no budget and no record of how many probes it made. The adversarial-generator experiment addresses this directly: without the enclave's round limit and feedback budget, $\Delta$ should grow with the number of probes. If it does not, the state machine adds little and the design simplifies to Proof-of-Guardrail plus a test runner.

Why not use zero-knowledge proofs or encrypted computation instead of hardware? Running test suites and interpreters inside such proofs is far slower. PRESAGE already reports a 120x speedup of enclaves over homomorphic encryption on a much simpler pipeline. Purely cryptographic verification remains the long-term answer to concerns about trusting the hardware vendor, but it is not a practical comparison point for multi-round agent loops today.

Proposed Experiments

The main experiment runs HumanEval, MBPP and their edge-case ET variants, plus CoderEval, under the hidden/shadow test split. For each of the five feedback policies and a range of round limits we report shadow Pass@1, the leakage gap $\Delta$ under honest and adversarial generators, end-to-end latency, enclave cost, and orchestration tokens as a share of the total. Orchestration tokens are compared against the 15% threshold from The Orchestrator Bottleneck. The MATH extension applies the same split to Level 5 problems. MACM-style conditions are checked by executed code inside the enclave, and the reference is the 54.68% to 76.73% jump from MACM. As a secondary, non-code check, rule matching on ComplianceReview tests whether a sealed proprietary rulebook behaves like sealed tests. No reported result was found on Lacuna for ComplianceReview under any confidentiality constraint.

Three outcomes would decide the matter. First, if binary or count-only feedback keeps most of the full-feedback gain on the ET suites while the adversarial $\Delta$ stays small under a fixed budget, the central claim holds. Second, if retained gain drops steeply as feedback detail drops, decomposition gains depend on leaking the verifier's knowledge. That negative result would bound what any confidential verification loop can achieve. Third, if per-candidate batching holds the boundary cost near the low end of the range AUTOTEE measured rather than the high end, the systems design is sound. If boundary cost dominates even after batching, the split must be coarsened to whole-task granularity, and the advantage over single-enclave wrapping narrows to the trusted-code-base and model-privacy arguments alone.