Invariant-Aware Harness Enforcement to Protect LLM Agents from Silent API Failures

Sajib Acharjee Dip, Brian Ouyang, Lala MotlhabiarXiv 2026 · arXiv

TL;DR

LLM agents often fail when APIs return "silent" errors—plausible but incorrect data delivered with a successful HTTP 200 status. This paper introduces SilentGuard, a system that moves verification logic from the LLM into the execution harness, using mathematical invariants to detect and block faulty tool outputs before the model ever sees them.

What Problem Does This Paper Address?

When an LLM agent calls an API, it assumes the result is correct if the call succeeds. However, APIs frequently suffer from silent failures. For example, if an agent sends a misnamed filter (e.g., organism=human instead of the documented human[orgn]), many production APIs will simply ignore the unrecognized parameter and return the unfiltered total count.

The agent receives a valid-looking number and a "200 OK" status, leading it to confidently report a hallucinated or irrelevant result. Previous attempts to fix this—like "LLM-as-a-judge" or self-correction checklists—rely on the model to catch the error. The authors discover a frustrating reality: LLMs are stubborn. Even when a secondary model flags a result as suspicious, agents often ignore the warning and report the faulty data anyway.

Methods

The authors propose SilentGuard, an "invariant-aware harness enforcement" system. Instead of asking the LLM to be skeptical, they build skepticism into the harness (the code that actually executes the API calls).

Key components include:

  • Exact Invariants: These are properties that must be true for any correct API. For example, the "Count Identity": the count of items matching a filter $F$ plus the count of items matching $\neg F$ must equal the total count. If this doesn't add up, the API is objectively failing.
  • Spec Checks: Automatically verifying that the agent's requested parameters exist in the API's documented schema.
  • Learned Contracts: For cases where mathematical invariants don't exist, the system mines "contracts" (e.g., expected data types or value ranges) from clean historical data.
  • Enforcement (The "Guard"): If a result fails an exact check, the harness withholds the value from the agent. The agent is told the result failed a consistency check and is prompted to either try a different approach or admit the answer cannot be determined.

Datasets

The study utilizes a mix of synthetic and real-world evaluations:

  1. AgentOrchBench (New): A benchmark created by the authors featuring 66 tasks (counting and listing) across eight local REST services. It allows for exact grading without needing an LLM judge.
  2. Real Public APIs: The authors probed 15 major research APIs (including NCBI, UniProt, and arXiv). They found that 11 of them exhibited silent failures, sometimes returning counts inflated by up to 691x.
  3. ToolMaze: An existing benchmark for multi-step tool workflows used to test how SilentGuard handles complex, interdependent tool calls.

Key Insights and Ideas

  • Warnings are not enough: In tests, agents reported a flagged (incorrect) count in 77.4% of the runs where a judge explicitly warned them the data was suspect. Enforcement—actually blocking the data—is the only reliable way to prevent the error.
  • The "Evidence-Permission-Enforcement" Triad: To stop failures, you need Evidence (a test showing the result is wrong), Permission (prompting the model that it is allowed to say "I don't know"), and Enforcement (the harness taking the decision away from the model).
  • Metamorphic Testing at Runtime: SilentGuard essentially performs metamorphic testing (checking relationships between different inputs/outputs) in real-time during the agent's execution.

Key Results

SilentGuard demonstrated massive improvements in reliability across various model classes:

  • Open-Weight Models: On AgentOrchBench, wrong answers dropped from 74.0% to 1.7%.
  • Frontier Models (GPT-4/5 class): Wrong answers fell from 70.2% to 0.5%.
  • Real-World Impact: On 9 public APIs with slightly incorrect documentation, SilentGuard cut wrong answers from 89.8% to 21.3%.
  • Zero Overhead on Success: The system maintained high accuracy on "clean" tasks (84.9%), meaning the safety checks didn't break the agent's ability to function normally when tools worked correctly.

Limitations and Future Work

  • Consistent Lies: If an API is "consistently" wrong (e.g., it lies about both the filtered count and the negated count in a way that satisfies the identity), SilentGuard cannot detect the failure. It requires an independent source of truth to catch "consistent" hallucinations.
  • Query String Complexity: The current system struggles to inspect filters hidden inside complex, unstructured query strings (like those used in some legacy bioinformatics APIs).
  • Abstention vs. Recovery: SilentGuard is currently better at abstaining (saying "I don't know") than recovering (fixing the call). Future work could involve the harness suggesting specific fixes to the agent based on why the invariant failed.

For more details, the code and AgentOrchBench dataset are available at github.com/Sajib-006/silentguard.

Summary updated

Abstract

LLM agents answer questions by calling APIs and trusting the results. An API can fail silently. It ignores part of a request, such as a misnamed filter, yet returns HTTP 200 and a plausible but wrong result. Current fixes, such as checklists, reviewer models and LLM judges, ask the model to catch it. But models often see the problem and answer anyway. We present SilentGuard, which moves the decision from the model to the harness running tool calls. It tests each result against invariants that any correct API must satisfy and withholds values that break one. We find silent failures in 11 of 15 public research APIs, with counts inflated up to 691 times. Current fixes leave over 70% of answers wrong, and agents report 77.4% of counts a judge flags. Across 7 open models, SilentGuard cuts wrong answers on its target faults from 74.0% to 1.7%, with no loss on fault-free tasks. On three frontier models they fall from 70.2% to 0.5%. A warning from the same check leaves 70.1% wrong, so enforcement is the key. On new fault types, the harness alone cuts wrong answers from 61.1% to 24.5%. It avoids wrong answers by abstaining rather than recovering.
BibTeX
@misc{dip2026invariant,
  author = {Sajib Acharjee Dip and Brian Ouyang and Lala Motlhabi},
  title = {{Invariant-Aware Harness Enforcement to Protect LLM Agents from Silent API Failures}},
  year = {2026},
  eprint = {2611.20595},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20595},
}
Figure