Targeting Intermediate Features for Transferable Adversarial Attacks

This direction is no longer present in the current build. Its last version stays readable and citable here.

Version 1 of 123 papers25 conceptsfrozen 2026-10-04

Most adversarial attacks are "skin deep." By optimizing perturbations to flip a model's final classification label, attackers often exploit idiosyncratic "bugs" or high-frequency noise patterns specific to a single model's weights. While these attacks are devastatingly effective against the model they were trained on, they often fail to transfer to other, unknown "black-box" models.

To solve this, a growing body of research suggests moving the battleground from the final output layer to the network's internal representations. The core intuition is simple: while different models (like a ResNet and a Vision Transformer) have different decision boundaries, they must all extract similar semantic features—like the shape of a wheel or the texture of fur—to recognize an object. By disrupting these shared intermediate features, attackers can create "universal" perturbations that fool almost any architecture.

Disrupting the Internal Representation

Standard attacks focus on the loss function at the very end of the network. In contrast, the Feature Disruptive Attack (FDA), introduced by Aditya Ganeshan et al. in FDA: Feature Disruptive Attack, aims to systematically corrupt features at every layer. This effectively "blinds" the model to the image content entirely, making the attack potent across diverse tasks like image captioning and style transfer.

Other methods focus on specific types of feature manipulation to enhance transferability:

Improving the Gradient Path

A major hurdle in intermediate attacks is the "vanishing" or "shattered" gradient problem caused by non-linear activations like ReLUs. If the gradient signal becomes too noisy as it backpropagates through the network, the resulting perturbation will be overfitted to the source model's specific non-linearities.

To fix this, Steven Y. Guo and Qizhang Li proposed Linear Backpropagation (LinBP) in Backpropagating Linearly Improves Transferability of Adversarial Examples. By treating the network as more linear during the backward pass (skipping the ReLU constraints), they generate smoother, more transferable gradients. Similarly, Jianping Zhang et al. used Neuron Attribution-based Attacks (NAA) in Improving Adversarial Transferability via Neuron Attribution-based Attacks to more accurately credit which internal features drive a decision, leading to more effective black-box perturbations.

The Layer Selection Problem

Despite the success of these methods, they all share a common Achilles' heel: sensitivity to layer selection.

As noted across nearly all the surveyed literature—from Yet Another Intermediate-Level Attack to the recent Pixel2Feature Attack (P2FA) by Renpu Liu—the effectiveness of a feature-space attack depends heavily on which layer is targeted.

  • Early layers capture low-level textures that might be too model-specific.
  • Late layers are too closely tied to the final classification logic, leading back to the overfitting problem.
  • The "Sweet Spot" varies by architecture. A layer that works for a ResNet-50 might be suboptimal for an Inception-v3 or a Vision Transformer.

Currently, identifying the optimal layer remains a manual, empirical process. Researchers like Jihan Yang in An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation and Huanrui Yang in DVERGE highlight that while we know intermediate features are the key to transferability, we lack a universal, architecture-agnostic heuristic for selecting them.

Beyond Attacks: Robustness and Generation

The principle of manipulating intermediate features isn't just for attackers; it's also a tool for defense and generative modeling.

In Robust SAM, Jiahuan Long et al. use Singular Value Decomposition (SVD) on specific intermediate layers to surgically harden the Segment Anything Model (SAM) against attacks. Meanwhile, in the realm of generative AI, Javad Rajabi et al. introduced Token Perturbation Guidance (TPG) in Token Perturbation Guidance for Diffusion Models, showing that shuffling intermediate token representations can improve the output quality of diffusion models without any retraining.

Whether the goal is to break a model, defend it, or improve its creativity, the consensus is clear: the most meaningful changes happen in the "hidden" layers where the model's true understanding resides.

Papers in this version

History & citationVersion 1 · 2026-10-04

No longer present in the current build · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_targeting_intermediate_features_for_tran_202fe291,
  title = {Targeting Intermediate Features for Transferable Adversarial Attacks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/targeting-intermediate-features-for-transferable-adversarial-attacks/rdr_202fe29148ca4fc1adf8ce46c8314fd2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_targeting_intermediate_features_for_tran_202fe291_v1,
  title = {Targeting Intermediate Features for Transferable Adversarial Attacks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/targeting-intermediate-features-for-transferable-adversarial-attacks/rdr_202fe29148ca4fc1adf8ce46c8314fd2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}