Visual reasoning models that score near-perfectly on benchmarks such as CLEVR (synthetic 3D scenes paired with questions about counting, spatial relations and attribute comparisons) can be broken without any pixel noise. Moving a few irrelevant "distractor" objects, in a way that leaves the correct answer unchanged, is enough. Making models robust to such attacks by adversarial training means searching for the worst scene edit inside every training step. That search is expensive, because each candidate edit must be re-rendered and passed through the whole vision pipeline.
We propose to move that search out of pixel space and into the small, structured space of object representations. In a model that first breaks an image into a set of objects and then reasons over pairs of them, "move a distractor" becomes "shift a few coordinates of one object vector." The worst such shift can be found with a handful of gradient steps through the small reasoning head alone. Our hypothesis is that this gives most of the robustness of rendered-scene adversarial training at a small fraction of its cost, and clearly more robustness than randomly moving distractors.
Core Claim
Measuring CLEVRness frames evaluation as a game. A reinforcement-learning agent relocates distractor objects in the scene graph, while rule checkers guarantee that the answer is preserved. Under this attack, Mdetr falls from 99.7% to 60%, SAN from 72% to 28%, and FiLM loses more than 12 points on average, while humans keep 91–95%. That paper is purely diagnostic and proposes no defense. Separately, adversarial training is costly because it solves a nested min-max problem. The file documents this cost in vision and in discrete text spaces (Towards Improving Adversarial Training of NLP Models).
Our claim is that for reasoners built on explicit objects, the threat in the CLEVRness study lives in a low-dimensional, mostly differentiable space. Adversarial training can therefore be run there directly:
$$\min_\theta \; \mathbb{E}_{(s,q,a)} \; \max_{\Delta \in \mathcal{S}(s,q)} \mathcal{L}\big(f_\theta(s \oplus \Delta, q),\, a\big)$$
The symbols are as follows. $s$ is the set of object vectors for a scene, $q$ the question, $a$ the answer, and $f_\theta$ the relational reasoning head. $\Delta$ is a displacement applied only to the position coordinates of objects that the question's program does not reference. $\mathcal{S}(s,q)$ is the set of legal displacements: the object stays in bounds, does not collide, and the answer is unchanged when the edited scene graph is run through CLEVR's symbolic program executor.
A Relation Network (A simple neural network module for relational reasoning) scores every object pair with a shared function and sums the results. Moving one distractor therefore changes $n-1$ pair terms at once, which is exactly the attack surface the CLEVRness agent exploits. Each inner step costs one backward pass through the head, on the order of $n^2$ small multilayer-perceptron calls. A rendered-scene loop costs a Blender render plus a full backbone pass per step.
Research Program
The work runs in two regimes of increasing realism.
In the first regime, the reasoner reads CLEVR's ground-truth scene descriptions. Object positions are then explicit and a slot-space edit is exactly a scene edit, so the method's core assumption holds by construction. We train Relation Networks and PrediNet (An Explicitly Relational Neural Network Architecture) under the objective above. PrediNet computes relations as vector differences between attention-selected object pairs. Including it tests whether the result depends on the summed-pairs design. Inner maximization uses projected gradient steps on distractor positions. After each step we run a cheap rejection test through the symbolic executor, which plays the role of the CLEVRness rule checkers.
In the second regime, the input is pixels. Following Toward Unsupervised Visual Reasoning, which found that frozen object-centric features (DTI-Sprites, 81.8% on CLEVR question answering) beat dense DINO features (60.4%), we freeze a DTI-Sprites object encoder and train only the relational head. Here the displacement acts on the encoder's inferred position parameters, which are an explicit part of each sprite's description in that model.
The open question in this regime is realizability: do slot-space adversaries correspond to failures on real rendered scenes? We test it directly. We take adversarial displacements found in slot space, write them back into the scene graph, render the scene, and measure how often the rendered image also fools the model. This is the step most likely to break the idea, and we treat it as a primary result rather than a footnote.
We also measure cost throughout. We report wall-clock time, total FLOPs, and the ratio of training cost to ordinary training for every method.
Baselines
The naive reference point is ordinary training of the same reasoner on clean CLEVR. Its relevant reported numbers are a Relation Network accuracy of 95.5% on CLEVR (from A simple neural network module for relational reasoning) and the CLEVRness drops listed above. Our first step is to re-measure our own clean-trained models against a freshly trained attack agent, so all comparisons share one attack implementation.
The strongest competitor is rendered-scene adversarial training. In this method the CLEVRness-style agent proposes distractor moves on training scenes, the scenes are re-rendered, and the model trains on the failures it finds. This is the most faithful defense against the threat and the method we aim to approach in robustness while undercutting it in cost. The file contains no reported result for this method: no reported result was found on Lacuna for adversarial training against the CLEVRness agent. We will therefore run it ourselves under the same architecture, data, and hyperparameter search budget, giving it as much compute as it needs to converge. Beating it means coming within a few points of its attacked accuracy at a training cost at least an order of magnitude lower.
The most dangerous cheap competitor is random distractor augmentation. It applies uniformly random legal displacements from $\mathcal{S}(s,q)$ in slot space, or equivalently random moves in rendered scenes. It is run at a compute budget matched to our method, so it sees more random samples per scene. Beating it requires a statistically clear gain in accuracy under the adaptive attack, especially on counting and existence questions, which CLEVRness identified as most vulnerable.
A third baseline isolates whether the gains come from finding worst-case moves or merely from asking the model to behave consistently. It is an intra-scene version of Latent Feature Relation Consistency. That paper adds a penalty that keeps the matrix of pairwise feature similarities across a batch unchanged under pixel attacks, giving small AutoAttack gains (for example +0.78% over standard adversarial training on CIFAR-10 with ResNet18), but it relies on full PGD training. We adapt the penalty to the matrix of within-scene pair scores under random distractor moves.
Finally, an ablation removes the executor-based constraint. Without it, the inner loop may change the true answer and teach the model wrong labels, so this ablation shows how much the answer-preservation check matters.
Why not just…?
Why not just augment with random distractor moves? This is the real threat to the proposal. It is cheap, and it may cover most of the vulnerable region if that region is large. Two pieces of evidence argue the other way. The CLEVRness agent finds failures through targeted search on models trained on broad scene distributions. In set representation learning, Adversarial Encoding Perturbation and Synthesis for Set Representation Auxiliary Learning reports that adversarial latent perturbation beats random element dropping (for example +9.56% Recall@20 on Friendster), although there it is used as a regularizer, not a defense. The compute-matched comparison above settles the question. If random augmentation matches us under the adaptive attack, we will report that the min-max step is unnecessary for this threat.
Why not just run standard pixel-space PGD adversarial training? It targets the wrong threat at the highest price. Bounded pixel noise does not describe relocating a whole object. The PGD inner loop multiplies backbone cost, and that cost is what makes defending large vision encoders impractical, as discussed in Robustness Tokens. We will include a single PGD-trained run only to confirm that its robustness does not carry over to distractor attacks.
Why not just canonicalize the scene, in the style of Normalize-and-Predict? Robust Learning against Relational Adversaries maps code to normal forms and so avoids adversarial loops, reporting roughly 40x lower cost. For code, the rewrite rules that preserve meaning are known in advance. In a visual question, which objects are irrelevant depends on the question, so a canonicalizer that removes distractors would already have to solve the reasoning problem. Sorting objects adds nothing, because Relation Networks are already insensitive to object order. We will report a question-agnostic canonicalization (sorting objects and quantizing positions to a grid) as a check.
Why not just add robustness tokens or a defensive prompt to the encoder? Robustness Tokens restores DINOv2 features under pixel attacks (63.1% versus 0.3% on ImageNet). The defensive prompt in Improving Adversarial Robustness in Vision-Language Models raises LLaVA-7B from 12.54% to 91.46% under APGD on COCO. Both defend the perception stage against pixel noise. A moved distractor is perceived correctly, and the failure lies in the reasoning over correct objects. The same paper also concedes that white-box attackers can fold the prompt into their attack.
Proposed Experiments
All experiments use CLEVR, which supplies ground-truth scene graphs and programs. Sort-of-CLEVR serves as a fast development set. The attack at evaluation is adaptive: a CLEVRness-style A2C agent is retrained from scratch against each defended model. Where the input is slot-based, we add a white-box gradient attack in slot space at test time, constrained to the same legal set $\mathcal{S}(s,q)$. We report clean accuracy, mean and worst-case accuracy under attack, per-question-type breakdowns, and the ratio of training cost to ordinary training.
Three checks decide the outcome. First, under the adaptive agent, slot-space adversarial training must beat compute-matched random augmentation, with no more than about one point of clean-accuracy loss. Second, in the pixel regime, a substantial share of slot-space adversaries must re-render into real failures. Robustness gained in slot space must also reduce the agent's success rate on rendered scenes. If either fails, slot space is not a faithful proxy, and the cheap loop is defending against an imaginary threat. Third, the method must come within a few points of rendered-scene adversarial training at a cost at least ten times lower.
If all three hold, the natural next test applies the same inner loop to the explicit scene-graph nodes produced by structured multimodal models such as Scene Graph Thinking. That would ask whether robustness to object-arrangement attacks can be bought cheaply at foundation-model scale too.
References
- Measuring CLEVRness: Black-box Testing of Visual Reasoning Models
- Towards Improving Adversarial Training of NLP Models
- A simple neural network module for relational reasoning
- An Explicitly Relational Neural Network Architecture
- Toward Unsupervised Visual Reasoning: Do Off-the-Shelf Features Know How to Reason?
- Latent Feature Relation Consistency for Adversarial Robustness
- Adversarial Encoding Perturbation and Synthesis for Set Representation Auxiliary Learning
- Robustness Tokens: Towards Adversarial Robustness of Transformers
- Robust Learning against Relational Adversaries
- Improving Adversarial Robustness in Vision-Language Models with Architecture and Prompt Design
- Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models