Moving target defense for embedded deep visual sensing against adversarial examples

Qun Song, Zhenyu Yan, Rui Tan

Introduction

Deep neural networks (DNNs) have become the backbone of visual sensing in autonomous systems, such as self-driving cars and advanced driver-assistance systems (ADAS). However, these models are notoriously vulnerable to adversarial examples—inputs modified by small, intentional perturbations that are often imperceptible to humans but cause the model to misclassify the object. A critical weakness in existing defense mechanisms is their static nature. Once a model is deployed, its parameters remain fixed, allowing an attacker to perform "white-box" analysis to craft highly effective, targeted attacks.

This paper introduces fMTD (fork Moving Target Defense), a strategy designed to protect embedded deep learning systems by introducing dynamism and unpredictability. By adhering to Kerckhoffs's principle—which suggests that a system should remain secure even if its inner workings are public, provided a secret key remains hidden—fMTD uses locally generated "quasi-secret" models to create a moving target that thwarts adversarial efforts.

The fMTD Framework

The core of fMTD is the dynamic generation and rotation of multiple "fork models." Instead of relying on a single static model, the system maintains an ensemble of models that are periodically updated to prevent attackers from gaining a stable target.

Overview of the fMTD framework including model generation, detection, and thwarting.

The framework operates through three main components:

  1. Fork Model Generation: To ensure efficiency on resource-constrained embedded hardware, fMTD does not train new models from scratch. Instead, it takes a factory-trained "base model," applies random perturbations to its weights, and performs rapid "in situ" retraining. This process creates fork models that maintain high accuracy on clean data but possess unique decision boundaries, making them resistant to attacks transferred from the base model.
  2. Ensemble Sensing and Detection: At runtime, an input image is processed by an ensemble of $N$ fork models. The system monitors the consistency of their outputs. If the models disagree, an attack is detected. This leverages the observation that adversarial examples typically have "minor transferability"; an attack optimized for one model is unlikely to fool a diverse set of fork models simultaneously.
  3. Attack Thwarting and Majority Rule: When an inconsistency is detected, the system uses a majority vote to determine the final classification. Because the failure modes of fork models under attack are often unpredictable and random, the majority of models are likely to retain the correct classification or fail in non-overlapping ways.

Implementation and Efficiency

A major challenge for ensemble-based defenses in real-time systems is computational overhead. Running dozens of deep models in parallel can exceed the power and latency budgets of embedded boards like the NVIDIA Jetson. To address this, the authors propose a Serial Execution mechanism with early stopping. By evaluating models sequentially and stopping once a sufficient confidence threshold is reached, the system achieves up to a 5x reduction in inference time compared to standard parallel execution, making it viable for high-speed autonomous sensing.

Experimental Evaluation

The fMTD approach was rigorously tested against standard benchmarks, including MNIST, CIFAR-10, and the German Traffic Sign Recognition Benchmark (GTSRB), which is particularly relevant for ADAS applications.

Visual comparison of clean examples and adversarial examples crafted against different target labels and confidence levels ($\kappa$).

The results demonstrate the robustness of the moving target approach:

  • Attack Success Rate (ASR) Reduction: Against the powerful Carlini & Wagner (C&W) attack, fMTD achieved a defense success rate of 98% on the CIFAR-10 dataset.
  • Detection Performance: With an ensemble of 20 fork models, the system reached a true positive rate for attack detection exceeding 90% across all datasets.
  • Clean Accuracy Preservation: Importantly, the defense does not degrade performance on legitimate inputs. In several tests, the ensemble approach slightly improved accuracy over the single base model due to the inherent benefits of model averaging.

Conclusion and Future Directions

fMTD represents a significant step toward securing autonomous visual sensing. By transforming the defense from a static barrier into a moving target, it forces attackers to contend with an unpredictable and ever-changing decision space. While the current work focuses on single-image classification, future research will explore extending these principles to object detection, image segmentation, and temporal consistency in video streams to provide comprehensive protection for autonomous vehicles.

Summary updated

BibTeX
@inproceedings{song2019moving,
  author = {Qun Song and Zhenyu Yan and Rui Tan},
  title = {{Moving target defense for embedded deep visual sensing against adversarial examples}},
  booktitle = {SenSys},
  year = {2019},
  doi = {10.1145/3356250.3360025},
}
Figure