Moving Target Defense for Deep Visual Sensing against Adversarial Examples

Qun Song, Zhenyu Yan, Rui Tan

Introduction

Deep neural networks (DNNs) have become the backbone of visual sensing in autonomous systems, such as self-driving cars and advanced driver-assistance systems (ADAS). However, these models are notoriously vulnerable to adversarial examples—inputs modified by small, intentional perturbations that are often imperceptible to humans but cause the model to misclassify the object. A critical weakness in existing defense mechanisms is their static nature. Once a model is deployed, its parameters remain fixed, allowing an attacker to perform "white-box" analysis to craft highly effective, targeted attacks.

This paper introduces fMTD (fork Moving Target Defense), a strategy designed to protect embedded deep learning systems by introducing dynamism and unpredictability. By adhering to Kerckhoffs's principle—which suggests that a system should remain secure even if its inner workings are public, provided a secret key remains hidden—fMTD uses locally generated "quasi-secret" models to create a moving target that thwarts adversarial efforts.

The fMTD Framework

The core of fMTD is the dynamic generation and rotation of multiple "fork models." Instead of relying on a single static model, the system maintains an ensemble of models that are periodically updated to prevent attackers from gaining a stable target.

Overview of the fMTD framework including model generation, detection, and thwarting.

The framework operates through three main components:

  1. Fork Model Generation: To ensure efficiency on resource-constrained embedded hardware, fMTD does not train new models from scratch. Instead, it takes a factory-trained "base model," applies random perturbations to its weights, and performs rapid "in situ" retraining. This process creates fork models that maintain high accuracy on clean data but possess unique decision boundaries, making them resistant to attacks transferred from the base model.
  2. Ensemble Sensing and Detection: At runtime, an input image is processed by an ensemble of $N$ fork models. The system monitors the consistency of their outputs. If the models disagree, an attack is detected. This leverages the observation that adversarial examples typically have "minor transferability"; an attack optimized for one model is unlikely to fool a diverse set of fork models simultaneously.
  3. Attack Thwarting and Majority Rule: When an inconsistency is detected, the system uses a majority vote to determine the final classification. Because the failure modes of fork models under attack are often unpredictable and random, the majority of models are likely to retain the correct classification or fail in non-overlapping ways.

Implementation and Efficiency

A major challenge for ensemble-based defenses in real-time systems is computational overhead. Running dozens of deep models in parallel can exceed the power and latency budgets of embedded boards like the NVIDIA Jetson. To address this, the authors propose a Serial Execution mechanism with early stopping. By evaluating models sequentially and stopping once a sufficient confidence threshold is reached, the system achieves up to a 5x reduction in inference time compared to standard parallel execution, making it viable for high-speed autonomous sensing.

Experimental Evaluation

The fMTD approach was rigorously tested against standard benchmarks, including MNIST, CIFAR-10, and the German Traffic Sign Recognition Benchmark (GTSRB), which is particularly relevant for ADAS applications.

Visual comparison of clean examples and adversarial examples crafted against different target labels and confidence levels ($\kappa$).

The results demonstrate the robustness of the moving target approach:

  • Attack Success Rate (ASR) Reduction: Against the powerful Carlini & Wagner (C&W) attack, fMTD achieved a defense success rate of 98% on the CIFAR-10 dataset.
  • Detection Performance: With an ensemble of 20 fork models, the system reached a true positive rate for attack detection exceeding 90% across all datasets.
  • Clean Accuracy Preservation: Importantly, the defense does not degrade performance on legitimate inputs. In several tests, the ensemble approach slightly improved accuracy over the single base model due to the inherent benefits of model averaging.

Conclusion and Future Directions

fMTD represents a significant step toward securing autonomous visual sensing. By transforming the defense from a static barrier into a moving target, it forces attackers to contend with an unpredictable and ever-changing decision space. While the current work focuses on single-image classification, future research will explore extending these principles to object detection, image segmentation, and temporal consistency in video streams to provide comprehensive protection for autonomous vehicles.

Summary updated

Abstract

Deep learning based visual sensing has achieved attractive accuracy but is shown vulnerable to adversarial example attacks. Specifically, once the attackers obtain the deep model, they can construct adversarial examples to mislead the model to yield wrong classification results. Deployable adversarial examples such as small stickers pasted on the road signs and lanes have been shown effective in misleading advanced driver-assistance systems. Many existing countermeasures against adversarial examples build their security on the attackers' ignorance of the defense mechanisms. Thus, they fall short of following Kerckhoffs's principle and can be subverted once the attackers know the details of the defense. This paper applies the strategy of moving target defense (MTD) to generate multiple new deep models after system deployment, that will collaboratively detect and thwart adversarial examples. Our MTD design is based on the adversarial examples' minor transferability to models differing from the one (e.g., the factory-designed model) used for attack construction. The post-deployment quasi-secret deep models significantly increase the bar for the attackers to construct effective adversarial examples. We also apply the technique of serial data fusion with early stopping to reduce the inference time by a factor of up to 5 while maintaining the sensing and defense performance. Extensive evaluation based on three datasets including a road sign image database and a GPU-equipped Jetson embedded computing board shows the effectiveness of our approach.
BibTeX
@inproceedings{song2019moving,
  author = {Qun Song and Zhenyu Yan and Rui Tan},
  title = {{Moving Target Defense for Deep Visual Sensing against Adversarial Examples}},
  booktitle = {SenSys},
  year = {2019},
  doi = {10.1145/3356250.3360025},
}
Figure