Matching Color Attacks to the Receptive Field They Target

People see fine detail in brightness but only coarse detail in color. A color change that no viewer notices therefore has to be spatially smooth: it spreads gently over tens or hundreds of pixels instead of flickering pixel to pixel. This proposal asks what such a smooth color change looks like to a vision network. The answer should depend on how much of the image each of the network's units can see, which is its receptive field. A convolutional network whose units see only a small patch receives an almost constant color offset in each patch. A network with global context, such as attention or very large kernels, can integrate the whole smooth field.

The core claim is that a model's vulnerability to imperceptible color perturbations should peak at a spatial scale set by its effective receptive field. The plan is to measure this with a controlled sweep. We bound the perturbation by a perceptual color-difference limit, vary its spatial scale, and test architectures that differ mainly in receptive field. If the claim holds, it settles an open question in the literature: whether widening receptive fields protects models against chrominance attacks or makes them more fragile. It also yields a practical attack and defense rule, which is to pick the color perturbation's scale to match, or to avoid, the target's receptive field.

Why this gap is real

Color-space attacks already exploit the eye's weakness for color. ColorFool shifts only the two color-opponent channels $a, b$ of the LAB color space, a space built so that equal distances look roughly equally different to people. It uses segmentation masks to keep skin and sky natural. Anti-Forgery places dense LAB $a,b$ perturbations on faces to disrupt DeepFake generators. Perception matters moves its perturbation budget from the brightness channel into the color channels of YCbCr, another brightness/color split. Backdoor work such as Chroma Backdoor hides triggers in the color channels of YUV. All of these motivate the proposal, and all share one limitation: they are tested on standard local CNNs such as ResNet and AlexNet, or on generators. None treats the perturbation's spatial scale as a variable to measure against the target's architecture. Anti-Forgery even notes that cropping partially neutralizes its perturbation, which hints that spatial extent matters.

Work on receptive fields supplies the architectures we will compare, each with its own way of seeing farther. CCNet attends along rows and columns, repeated twice, so every pixel reaches every other. Compact Generalized Non-local Network (CGNL) models interactions across positions and channels together. That makes it relevant here, because color information lives in specific input channels. ParCNetV2 uses one-dimensional kernels about twice the size of the feature map, which gives every unit a global view and breaks translation invariance. Butterworth as Attention gates frequencies globally after a Fourier transform, a learned filter over the image's spectrum. None of these papers evaluates adversarial or color robustness.

The closest bridge, Attention-Aware Temporal Adversarial Shadows, uses a pretrained transformer's attention maps as a loss term. It perturbs only brightness, never color, and does not ask how a target's receptive field shapes its vulnerability. EdgeFool likewise perturbs only brightness, with a local convolutional generator. These two papers constrain the design: they show that brightness-only attacks are already strong, so a color-only finding must be compared against a brightness-only control.

Core Claim

We parameterize a color perturbation by its spatial scale. Let $z$ be a free two-channel field over the image. The perturbation added to the LAB color channels is

$$(\delta_a, \delta_b) = G_\sigma * z \quad \text{subject to} \quad \Delta E_{00}(x, x+\delta) \le \tau \ \text{at every pixel},$$

where $G_\sigma$ is a Gaussian blur of width $\sigma$ pixels, $*$ is convolution, $\Delta E_{00}$ is the CIEDE2000 perceptual color difference, and $\tau$ is a fixed visibility budget. For a model $f$, let $V_f(\sigma)$ be the attack success rate at budget $\tau$, and let $r_f$ be its effective receptive field radius. We measure $r_f$ empirically as the spread of the gradient of a central output unit with respect to the input.

The hypothesis is that local CNNs show $V_f(\sigma)$ peaking near $\sigma \approx r_f$ and falling off for larger $\sigma$, because each unit sees an almost uniform tint that normalization layers partly absorb. Global-context models should instead stay vulnerable, or become more vulnerable, as $\sigma$ grows. Under a perceptual budget, large-$\sigma$ color fields are exactly the ones the eye tolerates most. The intuition behind that last step is that permitted color amplitude grows as spatial frequency falls, so a global integrator gets the largest usable signal. If this holds, wider receptive fields trade some fragility to local texture for fragility to imperceptible color fields. If it fails and the curves have the same shape across architectures, color vulnerability is a property of the training data, not of receptive field. That would also be worth knowing.

Research Program

The work has three parts.

The first part is a controlled architecture family on CIFAR-100, trained from scratch under one recipe, so that receptive field is the main difference between models. The family is a ResNet-50-style backbone, the same backbone with dilated final stages, with an inserted CGNL block, with recurrent criss-cross attention, with ParCNetV2-style oversized kernels, and with a Butterworth spectral gating module. We also include a small vision transformer. CIFAR-100 is chosen because the Butterworth model reports 81.31% accuracy there, which gives a reference point, and because the full grid is affordable. Clean accuracy is matched within about one point; where it is not, we report the gap.

The second part is an observational check on ImageNet-1K using public pretrained ResNet-50, ConvNeXt, and ParCNetV2 models, with ParCNetV2 reported at 84.6% top-1. This tests whether the trend survives at scale and at real image resolution, where the eye's tolerance for smooth color changes matters most.

The third part is a dense-prediction extension on Cityscapes with CCNet, reported at 81.9% test mIoU. It asks whether large-$\sigma$ color fields in one region corrupt predictions far away. This is the "propagation" side of the open tension in the literature.

For each model we sweep $\sigma$ over roughly 1 to 128 pixels at several values of $\tau$, optimizing $z$ by projected gradient descent through the LAB-to-RGB conversion. We run an identical sweep on the brightness channel $L$ alone as a control. We also run a small two-alternative forced-choice human study at selected $(\sigma, \tau)$ settings, to confirm that the budget is actually invisible and not merely small in $\Delta E$.

Baselines

The naive reference point has two parts. The first is clean accuracy. The second is random Gaussian color fields at each $(\sigma, \tau)$ setting, with no optimization. Together they show how much of the vulnerability comes from the attack and how much from the color shift itself.

The strongest existing method is ColorFool. It reports black-box success of up to 97% on ImageNet and keeps above 60% success under denoising and JPEG compression. We will run its public procedure on the same image subsets and every architecture, and record its achieved $\Delta E_{00}$ and its human detection rate. That lets us compare at equal perceived visibility rather than at ColorFool's own settings. The question is not whether our attack beats 97% on ResNet. It is whether ColorFool's success drops on global-context models while scale-matched fields do not.

The second strong baseline is dense LAB $a,b$ projected gradient descent with no scale constraint, in the style of Anti-Forgery. The file gives no reported result on Lacuna for this method on ImageNet classification. It gets the same iteration count and budget as our attack.

For transfer we also compare against EdgeFool, which reports 52–71% ResNet-to-AlexNet transfer, compared with 2–4% for Carlini-Wagner.

The key ablation removes the scale choice. We compare the attack at $\sigma = r_f$, set from the measured receptive field of a surrogate model, against the best fixed $\sigma$ chosen on held-out data and against the unconstrained-scale attack. The proposal wins if matching scale to the receptive field gives higher success, or higher transfer to a held-out architecture, at equal measured visibility. That should hold across the controlled family, not just one model.

Why not just…?

Why not just run existing color attacks on more architectures? That would produce a leaderboard without a mechanism. ColorFool and Anti-Forgery fix their spatial structure through segment masks or dense pixel noise. Any architecture differences they reveal would therefore mix together scale, amplitude, and segmentation errors. Existing color attacks are one point in our sweep, so this alternative is included in the proposal rather than competing with it.

Why not just widen the receptive field with dilated convolutions? CCNet reports that dilation samples sparsely and leaves gridding gaps. ParCNetV2 argues that large dilated or two-dimensional kernels fall short of a true global view. Both claims are about accuracy, not robustness, so we include the dilated network in the controlled family. If it behaves like the global models, the cheaper fix suffices, and we will say so.

Why not just low-pass or subsample the color channels at the input? This is a real threat for small-$\sigma$ attacks. Color subsampling of the kind JPEG uses removes exactly the high-frequency color triggers that work like Chroma Backdoor depends on. Our hypothesis predicts it does nothing against large-$\sigma$ fields, which survive subsampling intact; ColorFool's retained success under JPEG is consistent with this. We test the defense directly across $\sigma$. Converting inputs to grayscale removes color vulnerability by construction but costs accuracy. We report it as an upper bound on how much a model relies on color.

Why not just use RGB adversarial training? ColorFool and Anti-Forgery report that defenses tuned for $L_p$ perturbations in RGB pixel space, where most adversarial training happens, do not stop LAB perturbations. We will adversarially train one ResNet with RGB $\ell_\infty$ PGD and rerun the sweep. The test is whether that training flattens $V_f(\sigma)$ at large $\sigma$.

Decisive Checks

The claim stands or falls on three results. First, across the controlled family, the $\sigma$ at which $V_f$ peaks should rank-correlate with the measured $r_f$, while the brightness-only control shows a weaker or different relation. Second, at equal human-confirmed invisibility, scale-matched color fields should beat ColorFool and unconstrained LAB attacks on the global-context models, and should transfer better from a surrogate whose receptive field matches the target's. Third, input color low-passing should stop small-$\sigma$ attacks and fail on large-$\sigma$ ones. If the first check fails, the receptive-field explanation is wrong, and the study instead reports architecture-independent color fragility on matched models.