Format-Controlled In-Context Auditing of Vision-Language Models

When a vision-language model scores better on a benchmark after seeing a few worked examples in its prompt, that gain can come from three different sources. The model may have learned from the examples how the answer should be written. It may have been nudged into recalling a memorized answer from a benchmark that leaked into its training data. Or it may have actually inferred the task. Current evaluations report the sum of the three. We propose to separate them with two controls. The first is grammar-constrained decoding, meaning decoding in which a parser masks out every next token that would break a fixed output format, so the examples can no longer teach format. The second is matched "procedural twins": freshly rendered images with the same task structure but content that cannot exist on the web, so memorization is ruled out.

This decomposition leads to a sharper contamination detector. A recent text-only method detects memorized benchmarks by checking whether adding in-context examples raises the model's confidence in the correct answer, as on unseen data, or lowers it, as on memorized data. We hypothesize that in vision-language models much of the confidence boost on clean data is format learning. That would blur the detector's signal. Measuring the confidence shift only over grammar-permitted tokens should remove this confound. The note tests that claim, extends it to purely visual in-context models that paint their answers as images, and turns the result into an audit of how much reported multimodal few-shot learning is real.

Core Claim

Three lines of work motivate the proposal. VL-ICL Bench shows that few-shot gains on standard multimodal benchmarks mostly reflect format mimicry: brevity, capitalization, label style. When format is decoupled from content, much of the gain disappears, and many models get worse as the number of examples grows from 2 to 8. Both Text and Images Leaked! shows that multimodal benchmarks leak into training corpora, often through their text alone. It detects this by perturbing prompts and measuring accuracy drops, for example $\Delta = -5.3$ for Claude-3.5-Sonnet on ScienceQA and $-5.7$ for Phi-3-vision on Vintage. Detecting Data Contamination in LLMs via In-Context Learning, known as CoDeC, reports 99.9% dataset-level AUC on text models. It uses one signal: context raises confidence on unseen data and lowers it on memorized data.

These findings sit uneasily together. If in-context examples in vision-language models mainly teach format, then CoDeC's "confidence goes up on clean data" signal will be dominated by format learning in exactly this setting. Format learning helps clean and leaked items alike, which compresses the gap the detector relies on. Our claim is that renormalizing the likelihood over tokens a grammar allows removes the format component. It leaves a residual confidence shift that reflects content only.

For a sample with image-and-question input $x$, reference answer $y$ of length $T$, and demonstration context $c$, we define the format-controlled log-likelihood as

$$\ell_G(y \mid c, x) = \sum_{t=1}^{T} \log \frac{p(y_t \mid c, x, y_{<t})}{\sum_{v \in A_t} p(v \mid c, x, y_{<t})},$$

where $A_t$ is the set of tokens the output grammar permits at step $t$. The per-sample shift is $\delta_G = \ell_G(y \mid c, x) - \ell_G(y \mid \varnothing, x)$. The dataset contamination score is the fraction of samples with $\delta_G < 0$, mirroring CoDeC's statistic.

The prediction is directional. On guaranteed-clean procedural twins, the unconstrained shift should be clearly positive, while $\delta_G$ should be much smaller, because format was the main thing the examples supplied. On deliberately contaminated splits, $\delta_G$ should stay negative. The seen-versus-unseen separation should therefore widen under the constraint. If it collapses instead, that is also informative: it would mean CoDeC's signal in vision-language models is itself largely a format effect.

What Is New Relative to Closest Work

CoDeC is the method we reuse. It has been validated only on text and never interacts with constrained decoding; our contribution is the format control and its extension to images. MM-DETECT measures contamination through output perturbation and explicitly leaves open whether constraining the output space masks or exposes leakage. That question is our central ablation. VL-ICL Bench diagnoses format learning but proposes no remedy. We use grammar constraints as a measurement instrument rather than an accuracy trick. Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning supplies the decoding method: incremental parsers set the scores of illegal tokens to minus infinity at about 1–4 ms per token. That paper treats constraints as a way to raise validity (from about 64% to 100% on Penn Treebank parse trees) and accuracy. We instead use them to subtract format from an in-context effect. DyVal and CLEVR supply the procedural-generation idea behind our twins. Neither produces structured outputs from visual in-context prompts, and neither is used as a clean reference for calibrating a detector.

Research Program

Stage one: build the paired evaluation. For each source benchmark we write an output grammar and a procedural twin generator. The grammars cover answer-option letters for ScienceQA, relation triples in a small JSON schema for GQA-style spatial relations, and label sets for the VL-ICL Bench induction and binding tasks. The twins reuse the CLEVR rendering approach and the 42-object cutout scheme from Multimodal Function Vectors for Visual Relations. They generate relation, counting, and operator-induction items with the same answer grammar, at complexity levels in the spirit of DyVal. Twins are regenerated with fresh seeds at evaluation time, so they cannot have been seen in training.

Stage two: create ground-truth contamination. Dataset-level AUC needs known labels, so we plant them. We continue training open vision-language models (Qwen3-VL and OpenFlamingo, both used in the function-vector work, plus LLaVA-OneVision, a top performer in VL-ICL Bench) on a random half of each benchmark's test split. The other half stays unseen. This mirrors the "seen versus unseen" design CoDeC uses with pre- and post-cutoff data. We add two leakage conditions: image-and-answer leakage, and text-only leakage, which is the unimodal pathway MM-DETECT identified. Post-cutoff items from MMBench-Live serve as a naturally clean set for unmodified models.

Stage three: decompose few-shot gains. For each model and benchmark we compute accuracy at 0, 2, 4, and 8 examples under three conditions: unconstrained decoding on the original items, constrained decoding on the originals, and constrained decoding on the twins. The unconstrained-minus-constrained gap estimates format learning. The originals-minus-twins gap under constraint estimates the memorization-linked part. The constrained gain on twins estimates genuine induction. We report these three quantities, together with the contamination score, as the audit output.

Stage four: extend to pure visual in-context models. Pixel-painting models such as the MAE-VQGAN model of Visual Prompting via Image Inpainting predict a masked quadrant as discrete codebook tokens. That model was trained on 88,000 arXiv computer-vision figures, which plausibly include rendered Pascal and COCO examples. Here the "grammar" becomes a restriction of the codebook. At each masked position, only tokens that occur in the demonstration's output quadrant are allowed. For binary segmentation, these are the tokens of a two-color mask. We compute $\delta_G$ on the masked-quadrant likelihood, comparing a query-only canvas with a one-example canvas. We then compare Pascal-5i images against rendered twins built from the same object classes. This is the first test of whether CoDeC's effect exists at all outside text. It also tests whether the palette restriction plays the same role there, because without it the model can gain confidence simply by learning "output a flat two-color image."

Baselines

The naive reference point is the reported benchmark score itself: few-shot accuracy under unconstrained decoding, taken at face value with no contamination audit. We report it for every model and benchmark so the decomposition shows how much of that number survives. Surface-overlap filtering cannot run here because pretraining corpora are unavailable. Both CoDeC and MM-DETECT argue it misses paraphrased and cross-modal leakage anyway.

The strongest existing detector is CoDeC applied unchanged, using full-vocabulary log-likelihood with and without demonstrations. It reports 99.9% dataset-level AUC on text models. We found no reported result on Lacuna for CoDeC on vision-language or pixel-painting models, so that number is a ceiling from another modality, not a target in this one. We run CoDeC on the same models, the same planted splits, and the same demonstrations: the same number of examples, sampled with the same seeds from the unseen half. The comparison is fair because the only difference between CoDeC and our score is the renormalization over $A_t$. We beat it if our score has higher dataset-level AUC, and higher per-sample AUC, on the planted splits across at least two of three models. The margin must be clearly larger than seed-to-seed variation, which Mind Your Format shows can be large: a 0.17 standard deviation from template changes alone. For that reason every condition is averaged over several prompt templates.

The second competitor is MM-DETECT's perturbation test: option shuffling for ScienceQA, keyword slot-guessing for captioning. We run it on the same planted splits with its published perturbations. We report whether each method correctly ranks text-only leakage, the case MM-DETECT was built for. We would count a draw here as acceptable if our method remains stronger on image leakage.

Two ablations isolate the key ingredient. The first computes our score with a trivial grammar that allows every vocabulary token; this is exactly CoDeC and checks that gains come from the constraint. The second replaces the full structured grammar with option-restricted scoring, which renormalizes only over whole candidate answers. On multiple-choice tasks the two are equivalent. The comparison matters on relation-triple and segmentation outputs, where there are too many candidate answers to list.

Why not just…?

Why not just use MM-DETECT's perturbations, which already work on multimodal models? Perturbation measures whether accuracy drops when surface form changes. That test needs tasks with a shuffleable surface, and it cannot separate a model that memorized answers from a model that is merely sensitive to templates, as documented in Mind Your Format. It also gives no picture of how few-shot gains break down. MM-DETECT remains a real competitor on text-only leakage, and we run it head to head on the same planted splits. If it matches our method there, we will report that honestly and confine our claim to image-level leakage and pixel-painting models, where no textual perturbation exists.

Why not just evaluate on procedurally generated benchmarks and stop worrying about detection? Fresh benchmarks tell you how a model performs on new data. They do not tell you whether the existing public numbers everyone cites are inflated, or by how much. Fresh benchmarks alone also cannot separate format learning from induction. We still use procedural twins, but as the clean reference that calibrates the detector and anchors the induction estimate, not as a replacement for auditing.

Why not just add a natural-language format instruction instead of grammar-constrained decoding? An instruction reduces format errors without guaranteeing format, and the residual format errors are exactly what contaminates the confidence measurement. The grammar-constrained decoding paper reports that unconstrained prompting produced valid parse trees only about 64% of the time. VL-ICL Bench finds that added context can hurt vision-language models. We include an "instruction-only" condition in stage three. If it drives the unconstrained-minus-constrained gap to zero, our format-learning estimate is overstated and the constraint is unnecessary for the audit.

Why not just apply CoDeC unchanged? This is the most serious threat. If format learning turns out to affect clean and leaked data equally, it simply shifts both distributions and CoDeC's AUC would be unaffected. The first ablation settles this directly. If AUC does not improve, the contribution shrinks to the decomposition audit and to the first measurement of CoDeC's behavior on visual models, both of which remain useful.

Proposed Experiments and Decisive Checks

The main table reports, per model and benchmark (ScienceQA, GQA relations, VL-ICL Bench tasks, the synthetic relation set, and Pascal-5i for the pixel model): dataset-level and per-sample AUC for CoDeC, MM-DETECT, and our score on planted splits; the three-way decomposition of few-shot gain at 2, 4, and 8 examples; and validity rates under unconstrained decoding.

Four checks decide the proposal. First, on twins, the unconstrained confidence shift must be positive while $\delta_G$ is much smaller. If both are equal, format is not the confound and the core hypothesis fails. Second, AUC on planted splits must rise with the constraint relative to the unconstrained ablation. Third, the contamination score must rise with dose when we vary how many times the leaked half was seen during continued training. This rules out the score tracking something other than memorization. Fourth, for the pixel model, $\delta_G$ must separate Pascal-5i from rendered twins in the direction CoDeC predicts. If it does not, CoDeC's mechanism does not carry over to image generation, and we report that as a negative result that limits the method.