Silent Until Sparse: Backdoor Attacks on Semi-Structured Sparsity

Wei Guo, Maura Pintor, Ambra Demontis, Battista BiggioarXiv 2025 · DOI · arXiv

Researchers from the University of Cagliari have introduced a sophisticated security threat to modern AI deployment pipelines. Their paper, "Silent Until Sparse: Backdoor Attacks on Semi-Structured Sparsity," demonstrates how optimization techniques designed to accelerate models on hardware like NVIDIA GPUs can be weaponized to hide malicious behaviors.

The Core Threat: Silent Until Sparse (SUS)

As deep learning models grow in size, engineers increasingly rely on 2:4 semi-structured sparsity to maintain performance. This technique, supported by NVIDIA’s Ampere architecture, prunes two out of every four contiguous weights, effectively doubling inference speed with negligible accuracy loss.

However, this predictable optimization process creates a vulnerability. The Silent Until Sparse (SUS) attack remains completely dormant in a full-sized "dense" model but activates once the model is pruned for high-speed deployment. This allows a malicious model to pass standard security checks during the "Released Model" phase, only to become dangerous once deployed by the end-user.

Overview of the SUS attack: the backdoor is silent in the released model but triggers a misclassification (Vehicle to Boat) once pruned.

How the Attack Works

The SUS attack exploits the deterministic nature of 2:4 pruning through a two-phase training procedure:

  1. Backdoor Training: The attacker identifies the specific weights that will be retained after 2:4 pruning. They train these weights to recognize a specific trigger (e.g., a small pixel patch) and produce a target malicious output.
  2. Backdoor Hiding: To ensure the attack is "silent" in the dense model, the attacker fine-tunes the weights destined to be pruned. These weights are trained to "neutralize" the malicious signal. In the full model, the two sets of weights cancel each other out, resulting in normal behavior.
The SUS attack pipeline, showing the transition from backdoor training and hiding to the final deployed model.

Overcoming Defenses: SUS-F and SUS-R

The researchers developed two variants of the attack to ensure robustness:

  • SUS-F (Four): Designed for standard 2:4 pruning scenarios.
  • SUS-R (Row): A more advanced version that is "permutation-proof." Users often rearrange weights (permutations) to keep the most important ones during pruning. SUS-R imposes row-wise constraints that force the hardware to select the attacker's intended weights regardless of the user's optimization attempts.

Experimental Results

The researchers evaluated SUS across ten different model architectures, including ResNets, VGGs, DenseNets, and Vision Transformers (ViT), using benchmarks like MNIST, CIFAR-10, and TinyImageNet.

Key Findings:

  • High Success, Low Visibility: In dense models, the Attack Success Rate (ASR) was typically under 5%, making it indistinguishable from a clean model. Once pruned, the ASR jumped to over 95% across almost all tested architectures.
  • Defense Evasion: The attack successfully bypassed major backdoor detection frameworks, including Neural Cleanse (NC) and Artificial Brain Stimulation (ABS), because these tools only analyze the model in its dense, released state.
  • Robustness: The backdoor remained active even if the end-user performed additional fine-tuning on the pruned model to recover accuracy.

Implications for AI Security

The SUS attack highlights a critical gap in current AI security protocols. Most existing defenses audit models as they are provided by a vendor, not as they will exist after deployment-time optimizations.

This research suggests that security audits must evolve to include "post-compression" evaluations. As the industry moves toward automated deployment pipelines and model hubs, ensuring that a model remains safe after it is optimized for hardware is becoming as important as its initial safety check.

Summary updated

Abstract

Semi-structured (2:4) sparsity is a widely adopted pruning method in modern hardware and software ecosystems (e.g., NVIDIA Sparse Tensor Cores and PyTorch), achieving up to 2X faster inference and reduced memory footprint with negligible accuracy loss. It removes two out of every four contiguous weights, using permutations to ensure the largest-magnitude weights are retained. In this work, we show that this predictable mechanism can be exploited to design Silent Until Sparse (SUS), a novel compression-activated backdoor attack tailored to the 2:4 sparsity regime. SUS employs a two-phase training procedure that modifies (i) the weights that will be retained after pruning to embed the backdoor, and (ii) the weights that will be pruned to hide it in the dense model. SUS also provides formal guarantees that the attack will be successfully activated after sparsification. Experiments show that SUS is largely effective against semi-structured sparsification across both hardware-accelerated and software pipelines, outperforming existing compression-aware backdoor attacks, bypassing standard defenses, and even being robust to user-side fine-tuning.
BibTeX
@misc{guo2025silent,
  author = {Wei Guo and Maura Pintor and Ambra Demontis and Battista Biggio},
  title = {{Silent Until Sparse: Backdoor Attacks on Semi-Structured Sparsity}},
  year = {2025},
  eprint = {2509.08747},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.08747},
}
Figure