Sensitive-Sample Fingerprinting of Deep Neural Networks.

Zecheng He, Tianwei Zhang, Ruby B. LeeCVPR 2019 · OpenReview

Abstract: Numerous cloud-based services are provided to help customers develop and deploy deep learning applications. When a customer deploys a deep learning model in the cloud and serves it to end-users, it is important to be able to verify that the deployed model has not been tampered with. In this paper, we propose a novel and practical methodology to verify the integrity of remote deep learning models, with only black-box access to the target models. Specifically, we define Sensitive-Sample fingerprints, which are a small set of human unnoticeable transformed inputs that make the model outputs sensitive to the model's parameters. Even small model changes can be clearly reflected in the model outputs. Experimental results on different types of model integrity attacks show that we proposed approach is both effective and efficient. It can detect model integrity breaches with high accuracy (>99.95%) and guaranteed zero false positives on all evaluated attacks. Meanwhile, it only requires up to 103X fewer model inferences, compared with non-sensitive samples.

Introduction: The Integrity Challenge in MLaaS

As Machine Learning as a Service (MLaaS) becomes a standard for deploying deep learning applications, users increasingly rely on cloud providers for model hosting. This outsourcing introduces significant security risks. First, an adversary might inject a "Trojan" or "backdoor" into the model, causing it to behave normally on most inputs but fail maliciously when a specific "trigger" is present. Second, a dishonest provider might substitute a high-performance model with a compressed, lower-quality version to reduce operational costs.

A Trojaned DNN misclassifying authorized users when a specific trigger (glasses) is present.

Verifying model integrity in these scenarios is challenging because users typically only have black-box access, receiving only the model's output for a given input without visibility into the internal weights.

Methodology: Sensitive-Sample Fingerprinting

To address these challenges, we propose a methodology based on "Sensitive-Samples." These are inputs specifically engineered to make the model's output highly sensitive to its internal parameters.

1. Sensitive-Sample Generation

Unlike standard adversarial attacks that seek to change outputs for a fixed model, our approach uses gradient ascent to find inputs that maximize the gradient of the output with respect to the model weights. By maximizing this sensitivity, even minute changes to the model's parameters—whether from tampering or compression—will result in a detectable shift in output. To ensure these samples remain stealthy, we apply constraints so they appear as natural images to human observers.

Sensitivity vs. SNR over iterations and comparison between original images and generated sensitive-samples.

2. Maximum Active-Neuron Cover (MANC)

A single sensitive sample may only exercise a small subset of the network's neurons. To ensure comprehensive integrity verification, we developed the Maximum Active-Neuron Cover (MANC) algorithm. This greedy search approach selects a diverse set of fingerprinting samples that collectively activate as many neurons as possible, ensuring that modifications in any part of the network are likely to be detected.

Experimental Evaluation

We evaluated our methodology across diverse tasks, including face recognition (VGG-Face, AT&T), traffic sign classification (GTSRB), and general image classification (CIFAR-10). The approach was tested against various integrity breaches, including Trojan injection, poisoning, and unauthorized model compression.

Key Results

  • Detection Performance: The method achieved a detection accuracy of over 99.95% with zero false positives across all evaluated attacks.
  • Efficiency: Sensitive-sample fingerprinting requires up to 103× fewer model inferences than using natural samples to achieve the same level of confidence.
  • Robustness: The fingerprints remain effective even if an attacker attempts "adversarial fine-tuning" to bypass detection, as such attempts often cause the model to deviate even further from its original state.

Conclusion

Sensitive-sample fingerprinting provides a practical and highly efficient solution for verifying the integrity of remote DNNs. By leveraging the inherent sensitivity of neural networks to parameter changes at specific "edge cases," we can detect unauthorized modifications with high precision using a minimal number of queries. Future work will explore the development of dynamic fingerprints to counter adversaries who might attempt to detect and bypass static verification samples.

Summary updated

Abstract

Numerous cloud-based services are provided to help customers develop and deploy deep learning applications. When a customer deploys a deep learning model in the cloud and serves it to end-users, it is important to be able to verify that the deployed model has not been tampered with. In this paper, we propose a novel and practical methodology to verify the integrity of remote deep learning models, with only black-box access to the target models. Specifically, we define Sensitive-Sample fingerprints, which are a small set of human unnoticeable transformed inputs that make the model outputs sensitive to the model's parameters. Even small model changes can be clearly reflected in the model outputs. Experimental results on different types of model integrity attacks show that we proposed approach is both effective and efficient. It can detect model integrity breaches with high accuracy (>99.95%) and guaranteed zero false positives on all evaluated attacks. Meanwhile, it only requires up to 103X fewer model inferences, compared with non-sensitive samples.
BibTeX
@inproceedings{he2019sensitive,
  author = {Zecheng He and Tianwei Zhang and Ruby B. Lee},
  title = {{Sensitive-Sample Fingerprinting of Deep Neural Networks.}},
  booktitle = {CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=HslgWU6GlOTB},
}
Figure