In the digital world, fooling a neural network is trivial: a researcher can precisely manipulate individual pixels to flip a classification label. However, translating these "adversarial examples" into the physical world—such as printing a pattern on a T-shirt to hide from a surveillance camera—is notoriously difficult.
The primary obstacle is that the real world is not rigid. While early physical attacks focused on flat surfaces like billboards DeepBillboard or stop signs, humans and the clothes they wear are non-rigid. As a person moves, their clothing folds, wrinkles, and stretches, distorting any adversarial pattern printed on it. This research direction focuses on bridging the "digital-to-physical gap" by explicitly modeling these non-rigid deformations during the optimization process.
The Fragility of Static Patches
Early attempts at physical attacks, such as the adversarial patches developed by Simen Thys, Wiebe Van Ranst, and Toon Goedemé, demonstrated that a small, colorful cardboard sign could suppress the "objectness" score of a person detector Fooling Automated Surveillance Cameras. However, these patches were designed for rigid surfaces.
When Zuxuan Wu and Tom Goldstein attempted to create an "invisibility cloak," they found that the effectiveness of adversarial patterns significantly degraded when applied to fabric Making an Invisibility Cloak. The geometric instability of moving fabric creates a mismatch between the static digital simulation and the dynamic physical reality. To a neural network, a wrinkled patch looks like a completely different set of features than the flat patch it was trained to ignore.
Modeling the Physics of Fabric
To overcome this, researchers began incorporating differentiable models of deformation into the training loop. The intuition is simple: if you know how a pattern will warp, you can optimize it to remain adversarial across a wide range of possible distortions.
A landmark approach in this area is the Adversarial T-Shirt!, developed by Kaidi Xu, Sijia Liu, and colleagues. They utilized Thin Plate Spline (TPS) mapping to simulate how fabric wrinkles and folds as a person moves. By integrating TPS into the Expectation over Transformation (EoT) framework—which traditionally only handled rigid changes like lighting and rotation—they tripled the success rate of the attack in the physical world.
Similarly, Amira Guesmi and Muhammad Shafique introduced a "Creases Transformation" block in their DAP (Dynamic Adversarial Patch) framework DAP: A Dynamic Adversarial Patch for Evading Person Detectors. This mechanism randomly compresses pixel regions along specific axes during training, forcing the optimization to find patterns that are robust to the specific geometric signatures of fabric folds.
From 2D Patches to 3D Textures
As the field matured, the focus shifted from localized patches to full-body textures. Lifeng Huang and Alan Yuille proposed Universal Physical Camouflage (UPC), which models both external environmental factors and internal object deformations to hide entire categories of objects UPC: Learning Universal Physical Camouflage Attacks on Object Detectors.
Recent work has pushed this into the realm of 3D modeling. Zhanhao Hu and Xiaopei Zhu developed AdvTexture, which uses differentiable 3D meshes to apply adversarial signals across the entire surface of a garment Adversarial Texture for Fooling Person Detectors in the Physical World. By accounting for the wearer's body shape and gait, these textures maintain their "invisibility" even when viewed from 360 degrees.
The state-of-the-art in this direction, such as UV-Attack by Yanjie Li and Kaisheng Liang, leverages Neural Radiance Fields (NeRF) and UV mapping UV-Attack: Physical-World Adversarial Attacks on Person Detection via Dynamic-NeRF-based UV Mapping. By optimizing patterns against a statistical distribution of human poses synthesized via NeRF, they ensure the attack remains robust to complex, fluid human actions that traditional 2D augmentations cannot capture.
Expanding the Attack Surface
The principles of non-rigid modeling have also been applied beyond visible light. Xiaopei Zhu and Jianmin Li demonstrated that "Infrared Invisible Clothing" could hide wearers from thermal detectors by combining aerogel insulation with adversarial patterns optimized for fabric deformation Infrared Invisible Clothing: Hiding from Infrared Detectors at Multiple Angles in Real World.
Whether attacking face recognition with stickers that must warp over a curved forehead Robust Physical-World Attacks on Face Recognition or evading autonomous vehicle sensors AdvReal, the core insight remains the same: the physical medium is part of the model. By treating non-rigid deformation as a differentiable transformation rather than noise, researchers have turned the messy reality of the physical world into a predictable component of adversarial optimization.