When we optimize an adversarial perturbation against a specific neural network, we are navigating its loss landscape to find the minimal modification that pushes an input across a decision boundary. In a white-box setting where we have full access to the model's parameters, this optimization is highly effective. However, as noted by Jindong Gu et al. in A Survey on Transferability of Adversarial Examples Across Deep Neural Networks, these perturbations often fail completely when evaluated on a different, unseen target architecture. This drop-off in success is known as the adversarial transferability gap.
To understand why this gap exists and how researchers bridge it, we have to look at the geometric nature of how neural networks learn.
The Root Cause: Model-Specific Overfitting
Standard optimization techniques like Projected Gradient Descent (PGD) exploit high-frequency, highly idiosyncratic geometric features of a source model's decision boundary. These features are artifacts of a specific architecture's design choices—such as its layer depth, pooling mechanisms, or activation functions—rather than fundamental properties of the underlying data.
When an attack algorithm iteratively updates an image to maximize classification error, it naturally takes the path of least resistance, overfitting to these narrow, architecture-specific local minima. Consequently, an attack tailored to the structural quirks of a convolutional surrogate model misses the decision boundaries of a model with a different architecture, such as a Vision Transformer or a differently structured network.
To create transferable adversarial examples that work in true black-box scenarios, researchers must force the optimization process to ignore model-specific artifacts and instead target universal blind spots.
Breaking Alignment via Input Diversity
One of the most intuitive ways to prevent an adversarial perturbation from overfitting to a single model's architecture is to disrupt the precise pixel-to-neuron alignment during optimization. If a perturbation only works when an image is perfectly centered, scaled, and oriented, it is likely exploiting brittle, model-specific artifacts.
In Improving Transferability of Adversarial Examples With Input Diversity, Cihang Xie et al. introduced a mechanism that applies random transformations—such as resizing and padding—to the input image at each iteration of the attack.
Standard Attack:
[Input Image] ──> [Fixed Architecture] ──> [Overfitted Perturbation]
Input Diversity Attack:
[Input Image] ──> [Random Transform] ──> [Fixed Architecture] ──> [Generalized Perturbation]
By forcing the gradient updates to succeed across a diverse distribution of transformed inputs, the optimization process cannot settle into narrow, architecture-specific local minima. Instead, it is forced to find perturbations that are robust to spatial shifts, which naturally translate much better to the decision boundaries of unknown target models.
Targeting the Shared Feature Space
Another powerful intuition for improving transferability is to change where the attack is measured. Standard attacks optimize against the final output logits or loss function. However, the final decision boundaries of two different architectures can look radically different, even if they ultimately classify images correctly.
In contrast, the internal representations—the intermediate feature maps—often capture shared, fundamental characteristics of the visual world, such as edges, textures, and semantic parts. In MFA: Multi-layer Feature-aware Attack for Object Detection, Wen Chen et al. leverage this by optimizing perturbations against multiple internal feature layers simultaneously rather than just the final output. By distorting these foundational feature representations, the attack disrupts properties that almost any competent vision model relies on, bypassing the model-specific overfitting problem entirely.
Simulating Architectural Variance
If the goal is to make an attack work across many different architectures, an ideal strategy would optimize it against a diverse set of models simultaneously. While traditional ensemble attacks achieve this by averaging gradients across multiple pre-trained models, this approach is computationally expensive and limited to the specific models available during training.
To scale this intuition without the overhead of training multiple independent networks, Song Bai et al. proposed Ghost Networks in Learning Transferable Adversarial Examples via Ghost Networks. Instead of using multiple distinct models, the authors apply random feature-level perturbations to a single base network during the attack generation process, effectively creating an on-the-fly ensemble of "virtual" models.
Taking this a step further, Zheng Yuan et al. introduced the Meta Gradient Adversarial Attack, which uses a meta-learning framework to explicitly train the perturbation to generalize. By structuring each optimization step into a "meta-train" phase (on a surrogate setup) and a "meta-test" phase (simulating a black-box shift), the gradient updates are explicitly guided toward trajectories that remain effective across varying model behaviors.
The Challenge of Targeted Transferability
While causing a model to output any incorrect label (non-targeted transferability) is relatively straightforward, forcing an unknown target model to predict a specific incorrect class (targeted transferability) is exceptionally difficult. As early work by Yanpei Liu et al. in Delving into Transferable Adversarial Examples and Black-box Attacks showed, the specific gradient directions required to steer an input toward a precise target class are highly sensitive to an individual model's geometry.
To overcome this brittleness, Jacob M. Springer demonstrates in A Little Robustness Goes a Long Way: Leveraging Robust Features for Targeted Transfer Attacks that generating targeted attacks using a "slightly robust" source model significantly boosts cross-architecture success. By filtering out the highly fragile, non-transferable features and focusing the attack on robust, semantically meaningful features, the perturbation can successfully guide completely different architectures toward the desired target class.