Inference Time Randomization for Adversarial Robustness

20 concepts15 papers1 proposal

Deep neural networks are notoriously fragile. By applying a tiny, imperceptible perturbation to an input, an attacker can completely alter a model's prediction. Most adversarial attacks succeed because deep learning models are fundamentally deterministic static targets. Whether an attacker uses white-box gradients or black-box queries, they rely on a predictable, fixed decision boundary to find and exploit "blind spots."

Inference-time randomization flips this assumption. Instead of treating the model as a static landscape, this research direction introduces non-determinism directly into the inference phase. By transforming the model into a moving target, randomization disrupts the mathematical foundations that adversarial attacks depend on.

The Core Intuition: Shifting the Landscape

To understand why inference-time randomization works, imagine an adversarial attack as a hiker trying to find a hidden ravine (the adversarial blind spot) in a mountain range.

In a standard model, the landscape is frozen. A white-box attacker can calculate the exact slope of the mountain (the gradient) to walk straight into the ravine. A black-box attacker can throw out a few probes (queries) to map the terrain and locate the drop-off.

Deterministic Model:
Input (x) ---> [ Fixed Layers ] ---> Predictable Output (y)
                                      ^ Attacker easily maps this boundary

Randomized Model:
Input (x) ---> [ Stochastic Layer (Random Seed t1) ] ---> Output A
Input (x) ---> [ Stochastic Layer (Random Seed t2) ] ---> Output B
                                      ^ Boundary shifts with every query

When we introduce stochasticity during inference, the landscape constantly shifts. Every time the model processes an input—even the exact same image or text snippet—the internal coordinates or transformations change. The ravine is no longer where the attacker calculated it to be. The attacker steps toward their predicted blind spot, only to land on stable ground.

Disrupting the Optimization Loop

Adversarial attacks are optimization problems. White-box attacks use gradient descent (or ascent) to maximize the model's loss function relative to the input.

When noise modules or random sampling are active during inference, calculating a stable gradient becomes nearly impossible. As shown by Bo Huang et al. in Model-Agnostic Adversarial Detection by Random Perturbations, this stochastic environment prevents attackers from calculating reliable gradients. Because the specific noise injected changes with each forward pass, the gradient computed at step $t$ does not match the landscape at step $t+1$.

Ahmadreza Jeddi et al. leveraged this in Learn2Perturb: An End-to-End Feature Perturbation Learning to Improve Adversarial Robustness by training active noise-injection modules within the network's feature space. Because these modules remain dynamic at test time, they prevent adversaries from exploiting static decision boundaries. Similarly, in the audio domain, Yang Zhang et al. demonstrated in Voting for the Right Answer: Adversarial Defense for Speaker Verification that sampling multiple noisy versions of an audio clip creates a dynamic decision boundary that thwarts white-box optimization.

Black-box attacks face a similar barrier. These attacks rely on an iterative feedback loop, querying the model repeatedly to approximate the boundary. Noseong Park et al. broke this loop in SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert Patcher by randomizing the output layer using an ensemble of diverse "expert" heads. By returning inconsistent signals for the same input, the defense starves the attacker's optimization process of the stable feedback required to converge.

Where to Inject Stochasticity

Researchers have successfully integrated randomization at every stage of the inference pipeline, from the raw input to the deep latent layers and architectural routing.

1. Input-Space Transformations

The simplest approach is to alter the input before it propagates through the network. For computer vision, Cihang Xie et al. introduced a plug-and-play defense in Mitigating Adversarial Effects Through Randomization that randomly resizes and pads images. This breaks the precise pixel-level alignment that adversarial perturbations rely on.

In natural language processing, where inputs are discrete tokens, Xiaosen Wang et al. developed Detecting Textual Adversarial Examples through Randomized Substitution and Vote, which randomly swaps words with synonyms at inference time to break the fragile coordination of textual attacks. A similar philosophy underpins the work by Zhaoyang Wang et al. in RMLM: A Flexible Defense Framework for Proactively Mitigating Word-level Adversarial Attacks, which uses a randomized masked language model to disrupt the attacker's search space.

2. Feature-Space and Latent Perturbations

Instead of modifying the input, defenses can inject randomness directly into the hidden layers. In Adversary Resistant Deep Neural Networks with an Application to Malware Detection, Qinglong Wang et al. proposed Random Feature Nullification (RFN), which randomly masks features during inference. This forces the network to avoid relying on fragile, easily manipulated "blind spot" features.

3. Architectural and Routing Randomization

Rather than changing the data flowing through the network, some techniques change the network itself on a per-query basis. For example, Jialai Wang et al. developed Aegis: Mitigating Targeted Bit-flip Attacks against Deep Neural Networks, which randomizes early exit points in a multi-exit network to protect against hardware-level bit-flip attacks.

However, altering internal architecture can be a double-edged sword. As noted by Xuefei Ning et al. in Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness, randomizing internal path selection can cause fluctuations in clean accuracy, creating a trade-off between security and consistent performance.

The Limits of Randomization: Adaptive Adversaries

While inference-time randomization is highly effective against standard attacks, it triggers an evolutionary response from attackers: adaptive adversaries.

If a defense relies on simple randomness, a sophisticated attacker can bypass it using techniques like Expectation over Transformation (EoT), which averages gradients over multiple random states to find a perturbation that works across the board. As Carl-Johann Simon-Gabriel et al. demonstrated with their PopSkipJump attack in PopSkipJump: Decision-Based Attack for Probabilistic Classifiers, attacks can be designed to model classifier noise explicitly, proving that naive randomness offers a false sense of security.

To counter adaptive adversaries, modern stochastic defenses ensure that the randomness cannot be easily averaged out. For instance, Dong Lao et al. showed in Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensembles that latent ensembling remains resilient even when the attacker has full knowledge of the defense strategy.

By embedding the stochasticity deeply into the representation space or using randomized step sizes—as explored by Mingfu Xue et al. in Use the Spear as a Shield: An Adversarial Example Based Privacy-Preserving Technique Against Membership Inference Attacks—the defense ensures that the variance remains too high for an attacker to reliably reverse-engineer or bypass, preserving security in adversarial environments.

Key Foundational Work

Adversary Resistant Deep Neural Networks with an Application to Malware Detection (2017) — contributed 1 concept elements to this direction.
Mitigating Adversarial Effects Through Randomization (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_stochastic_inference_defense_34106a73,
  title = {stochastic inference defense},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stochastic-inference-defense/rdr_34106a7390a443c692482883b49933c4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_stochastic_inference_defense_34106a73_v1,
  title = {stochastic inference defense},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stochastic-inference-defense/rdr_34106a7390a443c692482883b49933c4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake