On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systems

Ivan Yu. Tyukin, Desmond J. Higham, Alexander N. Gorban

Abstract

This paper establishes a rigorous mathematical framework to demonstrate that adversarial examples are an inherent property of high-dimensional geometry and specific data distribution characteristics. The authors introduce the concept of "stealth attacks," showing how a single neuron modification can compromise a model's integrity for a targeted input while remaining invisible to standard validation protocols. By leveraging measure theory and stochastic separation theorems, the research proves that as dimensionality increases, the probability of both adversarial vulnerability and successful stealth insertion approaches certainty.

Introduction

Despite the transformative success of deep learning across various domains, AI systems remain notoriously brittle. Small, human-imperceptible changes to an input—known as adversarial examples—can trick state-of-the-art classifiers into catastrophic errors, such as misidentifying a "Stop" sign as a "Speed Limit" sign.

This research seeks to move beyond the empirical "cat-and-mouse" game of attack and defense. Instead, it addresses a fundamental question: Are adversarial examples an avoidable nuisance or a fundamental property of high-dimensional AI? Furthermore, the authors expand the threat model to include "stealth attacks," where a malicious insider can insert a "backdoor" that is mathematically guaranteed to evade detection through traditional black-box testing.

Theoretical Framework and Methods

The paper utilizes a formal approach based on high-dimensional geometry and measure theory to analyze model robustness. Two primary concepts drive the analysis:

  1. Smeared Absolute Continuity (SmAC): The authors define a property of data distributions where the probability density is not overly concentrated in infinitesimal regions. They demonstrate that if data follows this "smeared" distribution, adversarial examples are statistically guaranteed in high dimensions.
  2. Stochastic Separation Theorems: These theorems are used to prove that in high-dimensional spaces, a single target point (the attack vector) can be easily separated from a large set of other points (the validation set) using a simple linear boundary.

Stealth Attack Construction

The authors provide a mathematical recipe for a stealth attack. By adding or modifying a single Rectified Linear Unit (ReLU) or sigmoid node, an attacker can ensure the model produces a specific desired output for a chosen input $x'$ without changing the model's performance on a secret validation set $\mathcal{V}$. This single-node hijack acts as a precise switch that only activates for the attacker's specific target.

Geometric Insights and the Curse of Dimensionality

A central theme of the paper is how high-dimensional geometry favors the attacker. In high dimensions, the volume of a ball is concentrated almost entirely near its surface.

Definition of the unit ball $\mathbb{B}_n$ in $n$-dimensional Euclidean space.

This geometric "emptiness" makes it exponentially easy to find a point belonging to a different class just a tiny distance away from any valid data point. Theorem 1 demonstrates that the probability of an adversarial example existing approaches 1 exponentially fast as the dimensionality ($n$) of the system increases, provided the data distribution satisfies the SmAC condition.

Key Results

Adversarial Inevitability

The paper provides an explicit lower bound for the probability of adversarial examples. If the dimension $n$ satisfies $n > (\log \nu - \log C) [\log(1 - \epsilon/r_A)]^{-1}$, adversarial examples are expected to occur. This suggests that the "curse of dimensionality" is a primary driver of model vulnerability.

Stealth Attack Success and Validation Limits

Theorem 2 shows that a stealth attack succeeds with a probability of at least $1 - M(1/2\gamma)^n$, where $M$ is the size of the validation set.

  • The Validation Gap: If the dimensionality $n$ is high, even a validation set with millions of images ($M$) is statistically unlikely to catch a single-node modification designed to trigger on one specific target image. To truly detect a stealth attack, the validation set would need to be exponentially large ($2^n$), which is computationally infeasible for modern AI.

Conclusion and Future Work

The authors conclude that for a system to be truly robust, it must either have highly concentrated probability densities (essentially "memorizing" very specific points) or operate in a significantly lower-dimensional decision space. Traditional "black-box" validation is proven to be an insufficient defense against a malicious developer or a sophisticated adversary.

While the paper identifies the fundamental problem, it acknowledges limitations. The current framework focuses on targeted malevolent actions rather than random noise, and the probability estimates provided are lower bounds—meaning AI systems might be even more vulnerable than these formulas suggest. Future research must focus on developing "SmAC-resistant" architectures that can maintain integrity in high-dimensional feature spaces.

Summary updated

Abstract

In this work we present a formal theoretical framework for assessing and analyzing two classes of malevolent action towards generic Artificial Intelligence (AI) systems. Our results apply to general multi-class classifiers that map from an input space into a decision space, including artificial neural networks used in deep learning applications. Two classes of attacks are considered. The first class involves adversarial examples and concerns the introduction of small perturbations of the input data that cause misclassification. The second class, introduced here for the first time and named stealth attacks, involves small perturbations to the AI system itself. Here the perturbed system produces whatever output is desired by the attacker on a specific small data set, perhaps even a single input, but performs as normal on a validation set (which is unknown to the attacker). We show that in both cases, i.e., in the case of an attack based on adversarial examples and in the case of a stealth attack, the dimensionality of the AI's decision-making space is a major contributor to the AI's susceptibility. For attacks based on adversarial examples, a second crucial parameter is the absence of local concentrations in the data probability distribution, a property known as Smeared Absolute Continuity. According to our findings, robustness to adversarial examples requires either (a) the data distributions in the AI's feature space to have concentrated probability density functions or (b) the dimensionality of the AI's decision variables to be sufficiently small. We also show how to construct stealth attacks on high-dimensional AI systems that are hard to spot unless the validation set is made exponentially large.
BibTeX
@inproceedings{tyukin2020adversarial,
  author = {Ivan Yu. Tyukin and Desmond J. Higham and Alexander N. Gorban},
  title = {{On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systems}},
  booktitle = {IJCNN},
  year = {2020},
  doi = {10.1109/ijcnn48605.2020.9207472},
}
Figure