Tests That Compute Their Own Answers: Execution-Grounded Hallucination Suites for Vision-Language Models

Checking whether a vision-language model (a model that answers questions about images) makes things up requires test cases: an image, a question, and the correct answer. Writing those by hand is slow, which is why benchmarks go stale and mostly cover the easy case of "is there an X in the picture?" Letting a language model write the test cases instead brings back the problem the tests were meant to catch, because the model writing the answer key can hallucinate too. We propose to separate the two jobs. A language model writes a small program that draws the image, such as an HTML page, a plotting script, or an SVG diagram. It also writes a query function that reads the program's scene description and computes the expected answer. The answer key comes from running code, not from any model's belief about the picture. Small edits to the scene program, such as moving a button, recoloring a bar, or deleting an icon, produce families of related tests whose expected answers change in known ways.

The second half of the proposal addresses how to tell whether such a suite is any good. Software testing's standard answer is mutation testing: inject known bugs and count how many the tests catch. We carry this over by building "mutant" vision-language models with deliberately injected hallucination faults and scoring a suite by the fraction of mutants it exposes. Put simply: let code, not a model, supply the answer key, and grade hallucination benchmarks by how many planted faults they detect. Screenshots, charts, and diagrams are rendered from code in real life, so in these domains the approach has no synthetic-to-real gap. These domains are also where vision-language models are increasingly deployed inside software.

Core Claim

Execution-grounded oracles remove the main failure mode of LLM-written tests, the fabricated expected value, in visual domains whose images come from programs. Semantic scene mutations then produce tests with more fault-detection power per test than the negation and noise expansions or detector-filtered generation used today. Both claims can be measured: oracle correctness by human audit, and fault-detection power by the kill rate on mutant models.

How This Relates to Prior Work

The test-writing literature motivates the oracle design. CANDOR uses a panel of LLM agents to reach consensus on JUnit assertions from natural-language requirements. Nexus argues that agents sharing the same text priors reinforce one another's errors. It fixes this by running candidate assertions against an LLM-synthesized implementation in a sandbox, which raises LiveCodeBench oracle accuracy from 46.30% to 57.73%. We borrow the Nexus principle that execution, not agreement, should settle the answer.

Our setting differs in one important way. For images there is normally no implementation to run. Rendering from a scene program supplies one, and the scene program is the ground truth by construction rather than a plausible guess at it. This matters because POPE and MERLIM show that language co-occurrence priors override visual evidence. Any oracle built by LLM consensus, CANDOR-style, is therefore suspect for visual questions.

Three multimodal papers are the closest existing work, and each is a different way to automate hallucination tests. VHExpansion expands human-labeled seed triples by negating questions and perturbing pixels. It introduces symmetric accuracy, which gives credit only when a model answers both a question and its negation correctly. VHExpansion still depends on human seeds, and its expansions do not change what is in the scene. GHOST optimizes images in the embedding space of CLIP (a joint image-text encoder) so that they induce specific object hallucinations, and uses the OWLv2 object detector as the oracle. That detector is itself a learned model that can err. GHOST also covers only object existence. Drowzee generates metamorphic tests for text-only LLMs from knowledge-base triples using Prolog rules. We take its idea that logical relations between tests yield oracles for free and apply it to rendered scenes. Test-time consistency uses agreement across rephrasings and restylings as a training signal. That approach mitigates hallucination rather than measuring it, and it is complementary to ours.

On the evaluation side, MUTGEN shows that coverage can mislead: 100% coverage coexisted with a 4% mutation score. This is our reason to grade suites by mutant kills rather than by test counts or category coverage. SHARP finds directions in hidden activations that encode text bias versus visual conflict. We reuse that technique to plant controlled faults.

Research Program

The generator works in three domains. The first is user-interface screenshots rendered from HTML/CSS in a headless browser. The second is charts from a plotting library. The third is SVG diagrams such as flowcharts, floor plans, and icon grids. For each domain, an LLM writes a scene program that exposes a machine-readable scene description: elements, attributes, positions, containment, text, and counts. It then writes a query function over that description, in the style of "which button is directly left of the search box?", "how many bars exceed 40?", or "what color is the third node?". It also writes a natural-language question paired with that query.

A sandbox renders the image and runs the query to produce the expected answer. It then applies automatic validity checks. The queried element must actually be visible in the rendered pixels, which is checked from browser layout boxes or the SVG bounding geometry, so hidden or off-screen elements are excluded. The question must also be answerable, meaning the query returns a unique value. As in Nexus, failures feed back to the LLM for one refinement round.

Tests come in metamorphic families. Each mutation operator edits the scene program and declares how the answer should change. Deleting an element flips existence. Swapping two siblings flips left/right. Recoloring changes the attribute answer. Duplicating changes the count. Changing a nearby text label should leave the answer the same. The last operator provides "should be invariant" tests that target the shortcut behaviour MERLIM exposes by inpainting objects out. Here, however, removal is exact rather than inpainted. Each family is scored by a generalization of symmetric accuracy: a model gets credit only if it answers every member of the family correctly.

To grade suites, we build mutant models from open base models (LLaVA-1.5, LLaVA-NeXT, Qwen2.5-VL). Each mutant carries one of four fault classes, and each class comes in several strengths:

  • Existence yes-bias: steering activations along a SHARP-style text-bias direction.
  • Spatial binding faults: light fine-tuning on data with left/right labels swapped for a subset of object categories.
  • Attribute leakage: fine-tuning with colors swapped for chosen categories.
  • Reduced visual reliance: dropping or blurring a fraction of visual tokens.

A mutant counts as valid only if a held-out, hand-audited probe set shows that its behaviour really differs from the base model. This screens out "equivalent mutants", which are edits that change nothing observable. A suite kills a mutant if its accuracy on the mutant is significantly lower than on the base model under a paired test at fixed test budget. The suite's mutation score is the fraction of valid mutants it kills, reported both overall and per fault class.

Baselines

The naive reference is the static POPE suite scored by ordinary accuracy, run on the same models and mutants at a matched number of test items. POPE is what practitioners use today. Its reported weaknesses, a yes-rate near 99% for some models and an F1 drop from 78.86 to 71.37 for MiniGPT-4 under adversarial sampling (POPE), predict that it will kill existence mutants and miss spatial and attribute mutants.

The strongest existing test-expansion method is VHExpansion (VHExpansion). It has the reported result closest to our question: on POPE, symmetric accuracy reversed the ranking of Cambrian-1 and LLaVA-NeXT, 0.745 versus 0.798, where traditional accuracy gave 0.887 versus 0.879. We will run its negation and common-perturbation expansions on seeds drawn from our own domains, hand-labeling 200 seeds to match its setup, and score by symmetric accuracy. Its PGD adversarial expansion needs white-box gradient access and will be run only on the open models. Our suite beats it if, at equal test count, it kills significantly more valid mutants, particularly in the spatial and attribute classes. The mutants are held fixed across all suites. For absolute kill rates, no reported result was found on Lacuna for mutant-model kill rates of any hallucination suite, so all mutation-score comparisons are new measurements.

The strongest generative competitor is a GHOST-style pipeline (GHOST). Its reported hallucination-induction rate is 28–32%, against 0.1% for data-driven discovery, and it transfers from Qwen2.5-VL to GPT-4o 66.5% of the time. We compare using its released pipeline or a faithful reimplementation on COCO. Because GHOST targets natural-image existence, the comparison is on existence mutants and on oracle correctness, the latter audited by humans on a sample from each pipeline. GHOST's own human check found 89% of images truly lacked the target object, which implies roughly an 11% oracle error that our construction should avoid. Beating GHOST means lower audited oracle error at comparable kill rates on existence mutants, together with coverage of fault classes GHOST cannot generate.

The key ablation is the same pipeline with the executed answer replaced by an answer the LLM writes directly after seeing the scene program and rendered image, with the token budget matched. This isolates execution grounding, which is the proposal's central claim. A second ablation removes the metamorphic families and keeps only single tests, at matched count, to isolate the contribution of the mutation operators.

Why not just…?

Why not just prompt a frontier model like GPT-4o to write image-question-answer triples and have another model check them? This is the cheapest option, and it is exactly the setting in which Nexus reports that agents with shared priors reinforce one another's errors. MLLM judges are also documented to hallucinate defects (Chen et al.) and to shift by up to 26.5% across paraphrases (MERLIM). Our answer-key ablation measures this directly. If LLM-written answers on rendered scenes turn out to be nearly as accurate as executed ones, the oracle contribution collapses and the proposal reduces to its mutation-testing half. We will report that outcome if it occurs.

Why not just expand existing benchmarks with negation, as VHExpansion does? Negation removes guessing credit but leaves the image content unchanged, so it cannot test whether a model tracks a change in the scene. It also still requires human-labeled seeds. This alternative is a real threat on existence faults, where negated POPE may kill as many mutants as our suite. The per-fault-class mutation scores will show whether our advantage is confined to spatial, attribute, and count faults. If it is, we will say so.

Why not just use rendered synthetic scenes and skip natural images, or conversely, why trust results on rendered images at all? A reviewer may worry that findings on renders do not transfer. We restrict ourselves to domains whose real images are renders, namely UIs, charts, and diagrams. We also test transfer by fine-tuning on our generated tests and measuring change on POPE and MMVP, alongside the reported improvements from VHExpansion (0.180 to 0.711 symmetric accuracy) and GHOST (POPE 88.7 to 93.2). We make no claim about natural photographs beyond what that transfer shows.

Why not just deploy a runtime fix such as RITUAL or test-time consistency updates? These methods reduce hallucination but do not tell a developer whether a model is safe to ship or whether a model update introduced a regression. That is the job of a test suite. We include RITUAL (RITUAL) as a model under test to check whether our suite detects its gains.

Proposed Experiments

The first decisive check is oracle correctness. Annotators audit 1,000 sampled tests per pipeline, blind to which pipeline produced each test. We expect executed answers to be wrong far less often than LLM-written answers and detector-filtered answers. The second check is fault-detection power. We compute mutation scores for POPE, VHExpansion, the GHOST-style pipeline, our single-test ablation, and our full suite at test budgets of 500, 2,000, and 8,000 items, with per-class breakdowns and bootstrap confidence intervals. The third check is authoring cost. We compare human minutes and LLM tokens per valid test against the time to hand-label VHExpansion seeds, because the motivation is that writing tests is slow.

The fourth check is ranking validity. We rank LLaVA-1.5, LLaVA-NeXT, Cambrian-1, Qwen2.5-VL, and GPT-4o by each suite and compare against a hand-built gold set of 500 audited UI and chart questions. The proposal succeeds if our suite has fewer audited oracle errors than all baselines, kills significantly more valid mutants per test in at least the spatial and attribute classes, and produces model rankings that agree better with the gold set than ordinary accuracy on POPE.