Shading Banks: Training Wearable Adversarial Textures Against Real Fold Shadows

Adversarial clothing is printed fabric designed so that a person detector fails to see the wearer. Current methods train the pattern while simulating how the cloth bends and wrinkles. They then treat lighting as a global brightness and color shift. On a real body in direct sunlight, every fold casts a sharp shadow and every crease darkens. Glossy fibers add bright streaks that move with the viewing angle. These effects cut through the fine detail the pattern relies on. We propose to replace the color shift with physically correct shading on simulated, deforming garments. The trick that keeps this cheap is that a printed pattern changes only the cloth's base color (its albedo, meaning its reflectance under neutral light). So a rendered frame is close to linear in the texture: the image equals a per-pixel shading factor times the texture sampled through the fold, plus a highlight term that does not depend on the texture.

That linearity lets us move expensive light simulation out of the training loop. We simulate cloth offline and path-trace it, meaning we follow light rays physically through the scene, under many lighting conditions. We store each frame as a texture-coordinate map, a shading map, and a highlight map, and call this collection a "shading bank." The attack is then trained with ordinary gradient steps against frames drawn from the bank. The question we test is narrow and falsifiable. Does lighting that is tied to fold geometry close the gap between how well these attacks work indoors and how badly they degrade under harsh directional light? Or does cheap random shadow augmentation do just as well?

Core Claim

The physical weak point of wearable attacks is not fold geometry alone, which existing methods already model. It is how fold geometry interacts with directional light. Both shadows and fiber highlights are fixed by the same 3D fold that warps the pattern. A training signal that keeps the warp and the shading tied together should produce textures whose attack success holds up as light becomes more directional. Untied photometric augmentation should not achieve this. The new piece is not differentiable rendering in general. It is the observation that, for pigment-only attacks, texture-independent shading can be precomputed exactly with full light transport and reused, which keeps training costs at the level of current 2D methods.

Relation to Prior Work

Four attack papers motivate the gap, and each models deformation without physical light. Adversarial T-Shirt inserts thin-plate-spline warps, which are smooth 2D image distortions fitted to tracked wrinkles, into Expectation over Transformation. That method trains against randomly sampled distortions, and its color handling is a learned polynomial. DAP adds a block that simulates creases. AdvCaT uses 3D garment meshes with seam-consistent projection, but the texture remains an uncalibrated RGB albedo. UPC reports that its camouflage degrades in low light and under sharp shadows, which is direct evidence for our hypothesis. We keep their warping machinery and change only how pixels are lit.

The closest bridging work is UV-Attack. It uses a dynamic radiance field that outputs texture coordinates on a moving human, and it bakes the captured subject's lighting into appearance. Its 20–35% drop in success across subjects is consistent with that baked appearance overfitting. We share its "render coordinates, then sample the texture" structure but separate shading from albedo explicitly.

Adversarial Geometry and Lighting supplies the principle that physical image-formation parameters matter. Training with its spherical-harmonic lighting raised real-photo accuracy from 4.6% to 65.8%. Its renderer, however, is matte-only, rigid, and models no cast shadows, which are exactly the effects we add.

The inverse-rendering papers supply method and act as a constraint. ExMesh++ shows that modeling one-bounce indirect light on frozen geometry keeps shading out of albedo. It reports 34.19 dB relighting PSNR on Synthetic4Relight, against 24.22 dB for a variant without that transport. TensoIR ray-marches visibility for the same purpose. Both assume rigid scenes, so we do not use them on moving people. We use them on a static, mannequin-mounted printed garment to fit the fabric's roughness and sheen, which then calibrates the material model in the shading bank. CLOTH3D supplies over 7,000 sequences of physically simulated garments in cotton, silk, denim, and leather, which provide the deforming geometry.

Research Program

Construction proceeds in three steps.

First, we fit a sheen-plus-microfacet material for the actual printing substrate. We print a calibration chart on the target fabric, photograph it on a static mannequin from many views, and fit an ExMesh++-style decomposition to recover roughness and specular response. This number is measured once.

Second, we build the shading bank. We drape CLOTH3D T-shirt and top sequences on varied body shapes and path-trace them, including shadows and one-bounce indirect light. Lighting is sampled across a controlled range, from near-uniform overcast to a single hard sun at varied elevations. For each frame we store the texture-coordinate map, a diffuse shading map $S$, a highlight map $R$, and the garment mask.

Third, we train the texture $T$ with the composite image

$$I = S \odot \mathrm{sample}(T, u) + R,$$

where $u$ is the stored texture-coordinate map and $\odot$ is per-pixel multiplication. The composite is pasted onto backgrounds and passed through the detector's objectness loss. The only approximation is color bleeding from the pattern itself onto neighboring folds. We will measure it by comparing composites against full re-renders of trained textures, and report the PSNR.

Baselines

The naive reference is the same 3D-warped training with standard brightness and contrast jitter and no shadows. This is effectively the AdvCaT/Adversarial T-Shirt lighting treatment, re-implemented on our CLOTH3D geometry.

The strongest existing methods are AdvCaT and UV-Attack. For AdvCaT, the reported result is 85.94% physical attack success against Faster R-CNN (AdvCaT). For UV-Attack, the reported results are 92.7% digital success against Faster R-CNN and 85% physical success in free-pose tests (UV-Attack). Both numbers were measured under each paper's own lighting, and none is broken down by lighting directionality; we found no reported result on Lacuna for lighting-stratified attack success of any wearable attack. We will therefore re-run AdvCaT's open texture parameterization through our pipeline. We will use the same detectors, the same garment geometry, the same number of optimization steps, and the same Voronoi texture parameterization, so that only the lighting model differs.

The key ablation is geometry-decoupled shading. It uses the same shading and highlight maps, but each map is drawn from a different frame than the warp it is paired with. This preserves shading statistics while breaking the tie to geometry. Beating it isolates our central claim.

Winning means two things. Under hard sun, we must show higher attack success than every baseline with non-overlapping confidence intervals. Under overcast light, we must show no loss relative to them.

Why not just…?

Why not just add random shadow masks and gamma shifts to standard EoT? This is the most serious threat. The file's evidence against 2D augmentation, the roughly 30% drop when the warp is removed in Adversarial T-Shirt, concerns geometry, not lighting. So nothing yet rules it out. We include it as a baseline with shadow masks whose sizes and edge sharpness are matched to the bank's statistics. If it equals our method under hard sun, the conclusion is that tying light to geometry is unnecessary, and we will report it that way.

Why not just train on a dynamic radiance field captured outdoors, as UV-Attack does? That bakes one subject's light and folds into appearance. It cannot vary sun direction independently of pose. Its reported 20–35% cross-subject drop suggests it overfits. Our cross-body, cross-light split tests whether separating shading from albedo reduces that drop.

Why not just use low-frequency, natural-looking patterns that shadows cannot destroy? This may partly work: large blobs survive shadow edges. We will train both high- and low-frequency parameterizations under each lighting model. We will then report how much attack success each loses as the light becomes harder. If low-frequency textures already lose nothing, our contribution shrinks to the high-frequency regime, and we will say so.

Proposed Experiments

The digital test uses held-out CLOTH3D bodies, motions, and light directions, plus AttackScenes. AttackScenes provides controllable lighting and the evaluation of Faster R-CNN, YOLOv3, and SSD that is reported in UPC. On it we measure attack success and detector recall as functions of sun elevation and shadow fraction. Transfer to an unseen YOLO version is reported separately.

The decisive physical check prints each method's texture on the calibrated fabric. Several wearers of different builds walk under a lamp at three fixed angles and outdoors under overcast versus direct sun. We report success per lighting condition.

The claim holds if three things are true together: the gap between overcast and hard-sun success shrinks relative to AdvCaT's lighting model, the geometry-decoupled ablation reopens that gap, and the composite approximation stays close to full re-renders.