Gradient Alignment in Adversarial Transferability

This direction is merged into another direction: adversarial transferability. Its last version stays readable and citable here.

Version 1 of 134 papers49 conceptsfrozen 2026-10-04

In the study of neural network security, one of the most striking phenomena is adversarial transferability: a "poisoned" input designed to fool a specific model often successfully fools other, entirely different models that the attacker has never seen. This property enables "black-box" attacks, where an adversary crafts an exploit on a local surrogate model to compromise a remote, proprietary system.

Research across the field suggests that this transferability is not a coincidence of architecture, but a fundamental consequence of how different models learn to represent the same data. The core mechanism driving this phenomenon is gradient alignment.

The Geometry of Transfer

To understand why attacks transfer, we must look at the loss landscape of neural networks. An adversarial example is created by following the gradient of a loss function to find a small perturbation that flips a model's prediction.

Early work by Yanpei Liu and Dawn Song in Delving into Transferable Adversarial Examples and Black-box Attacks discovered a crucial distinction: while the specific gradient directions of different networks might appear nearly orthogonal (pointing in different directions), their decision boundaries often align in high-dimensional space.

This means that while two models might disagree on the exact "steepest" path to an error, they generally agree on which region of the input space constitutes a different class. Ambra Demontis and Battista Biggio formalized this in Why Do Adversarial Attacks Transfer?, identifying that attack success hinges on the cosine similarity between the gradients of the surrogate and the target. If the gradients point in a consistent direction, the perturbation is likely to cross the decision boundaries of both models.

Finding the Consensus Direction

If transferability depends on gradient alignment, the attacker’s goal is to find a "universal" direction that is shared by as many models as possible. Several strategies have emerged to identify this consensus:

  1. Ensemble Alignment: By maximizing the cosine similarity between the gradients of multiple surrogate models, researchers like Ziruo Wang in the Similar Target (ST) method force the attack to ignore architecture-specific quirks. This identifies fundamental vulnerabilities common to diverse architectures, such as CNNs and Transformers.
  2. Meta-Learning: The Meta Gradient Adversarial Attack (MGAA) by Zheng Yuan treats transferability as a learning task, iteratively simulating black-box scenarios to find perturbations that remain effective across diverse model families.
  3. Early-Stage Features: Interestingly, Chris Miller found that undertraining a surrogate model can actually boost transferability. Early in training, models focus on coarse, universal features before overfitting to specific architectural nuances. These early-stage gradients are often more aligned with the gradients of other models than the "refined" gradients of a fully converged network.

The Decision Boundary Advantage

A key insight for boosting transferability is where the gradients are calculated. Standard attacks calculate gradients at the original image coordinates. However, Lianli Gao and Qilong Zhang demonstrated in their Boundary Fitting Attack (BFA) that gradients become significantly more similar between different architectures as you approach the decision boundary. By refining the attack path to stay near these classification thresholds, attackers can exploit the regions where model sensitivities are most likely to overlap.

Perhaps the most profound implication of gradient alignment is its connection to how models learn. Kaizhao Liang and Bo Li established a bidirectional link between adversarial transferability and knowledge transferability.

They proved that if an attack transfers well from Model A to Model B, it is a strong mathematical indicator that Model A is a good candidate for transfer learning to Model B's domain. This suggests that adversarial vulnerabilities are not just "bugs," but are intrinsically tied to the representational alignment of the models. When two models learn similar functional mappings from inputs to outputs, they naturally develop similar gradient structures—and thus, share the same blind spots.

Defending through Misalignment

Understanding that alignment is the root of the problem provides a roadmap for defense. If an attacker relies on the consistency of gradient structures, a defender can succeed by intentionally disrupting that consistency.

For example, Amira Guesmi and Muhammad Shafique proposed TriQDef, which prevents attacks from transferring across different quantization levels (e.g., from a 32-bit model to an 8-bit model) by intentionally misaligning the internal feature and gradient structures. By breaking the "consensus" that attackers rely on, defenders can create models that are functionally similar but adversarially distinct.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_gradient_alignment_in_adversarial_transf_4fea7cec,
  title = {Gradient Alignment in Adversarial Transferability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/gradient-alignment-in-adversarial-transferability/rdr_4fea7cec00c049828d2026f54719f53a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_gradient_alignment_in_adversarial_transf_4fea7cec_v1,
  title = {Gradient Alignment in Adversarial Transferability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/gradient-alignment-in-adversarial-transferability/rdr_4fea7cec00c049828d2026f54719f53a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}