Most adversarial attacks are designed to flip a model's final decision—for example, making a classifier label a "panda" as a "gibbon." However, a more potent and stealthy class of attacks targets the internal "perception" of the network. Instead of manipulating the output label, these attacks force the neural network’s internal activations to mimic the representation of a completely different target image.
The Shift from Labels to Latents
In a standard adversarial attack, the goal is to cross a decision boundary. In contrast, feature-space mimicry aims to move a source image's position within the model's hidden layers until it resides in the same mathematical neighborhood as a target image.
Sara Sabour, David J. Fleet, and their colleagues demonstrated this in Adversarial Manipulation of Deep Representations. By applying imperceptible perturbations to an input, they could minimize the Euclidean distance between the internal activations of a source image and a "guide" image. To the human eye, the image remains unchanged; to the network's internal layers, the two images become virtually indistinguishable.
This approach, often called Layerwise Origin-Target Synthesis (LOTS), was further formalized by Andras Rozsa, Manuel Günther, and Terrance E. Boult in Exploring LOTS in Deep Neural Networks. They discovered a counter-intuitive vulnerability: systems that rely on "deep features" (like biometric face matchers) are often easier to deceive than standard classifiers because their internal representations are highly malleable.
Why Feature Mimicry Works
The intuition behind this attack lies in the geometry of high-dimensional latent spaces. Deep neural networks compress complex input data into semantic vectors. Because these vectors represent the "essence" of an object (e.g., the features that make a face look like a specific person), aligning these vectors allows an attacker to hijack the system's identity logic.
This is particularly dangerous in retrieval-based systems, such as person re-identification (re-ID) used in surveillance. Zhibo Wang and Hairong Qi showed in advPattern that physical-world garments can be designed to manipulate these feature extractions. By wearing a specific pattern, an individual can force the re-ID system to generate a feature vector that matches a different person in the database, effectively "impersonating" them in the latent space.
Superior Transferability Across Architectures
One of the greatest challenges for adversarial attacks is "transferability"—an attack crafted for one model often fails on another. Feature-space attacks overcome this by targeting semantic concepts rather than specific decision boundaries.
Jacob M. Springer and Melanie Mitchell found that by optimizing perturbations to mimic internal representations in a "slightly robust" model, the resulting attacks become much more effective at fooling entirely different architectures. In A Little Robustness Goes a Long Way, they showed that attacks generated on standard CNNs could successfully transfer to Vision Transformers (ViT) and CLIP. Because different models often learn similar semantic features to solve the same task, an attack that successfully mimics a feature in one model is likely to do so in another.
Identifying Hidden Models
The unique "representational fingerprint" of these internal layers can also be used defensively or for reconnaissance. Shaopeng Fu and Di Wang introduced the Pre-trained Encoder Inference (PEI) attack in Pre-trained Encoder Inference.
By sending synthetic inputs to a black-box service and analyzing the resulting outputs, an attacker can determine which specific pre-trained encoder (like a specific version of ResNet or a Vision Transformer) is being used upstream. This works because each architecture has a distinct way of mapping inputs to feature space; by finding which candidate model's internal representations most closely match the service's behavior, the attacker can unmask the hidden architecture, making subsequent feature-mimicry attacks even more precise.