TL;DR
This paper introduces Michscan, a novel method for detecting integrity attacks on TinyML models running on resource-constrained devices. By analyzing the instantaneous power consumption of the device, Michscan can identify malicious modifications to the model without requiring access to its internal parameters or the cooperation of the model owner, addressing a critical security gap in model licensing scenarios.
What Problem Does This Paper Address?
As neural networks become increasingly integrated into critical decision-making systems, especially on edge devices (TinyML), ensuring their integrity is paramount. Integrity attacks, such as Trojan attacks, data poisoning, and fault injection, can subtly alter a model's behavior, leading to incorrect classifications or performance degradation. This problem is exacerbated in scenarios where end-users license pre-trained models from third parties. In these "black-box" scenarios, the model owner often protects their intellectual property by withholding model parameters. This lack of transparency, combined with the potential unwillingness of model owners to cooperate on integrity checks (to avoid revealing sensitive information), leaves end-users vulnerable. Existing integrity-checking methods typically require white-box access to model parameters or a cooperative model owner, making them unsuitable for these prevalent licensing situations.
Methods
Michscan leverages power analysis to detect integrity violations. The core idea is that modifications to a neural network's parameters will alter its computational flow, which in turn affects the instantaneous power consumption of the device running the model.
The methodology involves two main stages:
- Pre-Deployment Stage (Controlled Environment):
- Template Generation: A "golden template" of the model's power consumption is created when the model is known to be in a benign state. This involves collecting multiple power traces during inference with a specific, consistent test input.
- Filtering: To combat noise in power traces, a frequency-domain analysis is performed to identify a "systematic variation" component. A Butterworth band-pass filter is designed based on this analysis to isolate this component.
- Similarity Sample Generation: The filtered power traces (excluding the one used for the golden template) are correlated with the golden template. The resulting set of similarity values forms a "similarity sample," representing the distribution of benign behavior.
- Runtime Analysis Stage (Untrusted Environment):
- Test Trace Collection: A small number of power traces (NRA) are collected during runtime inference using the same test input as in the pre-deployment stage.
- Filtering and Correlation: These test traces are filtered using the pre-determined filter and then correlated with the golden template to generate "test similarities."
- Hypothesis Testing: A Mann-Whitney U-Test is employed to statistically compare the distribution of test similarities against the similarity sample. This non-parametric test determines the probability (P-value) that the observed differences are due to random chance or a genuine integrity violation. If the P-value falls below a user-defined threshold (Pth), the null hypothesis (that the distributions are the same) is rejected, indicating a likely integrity violation.
Datasets
The paper evaluates Michscan using four TinyML models from the MLPerf Tiny benchmark suite:
- Keyword Spotting (Dataset: Speech Commands)
- Visual Wake Words (Dataset: Visual Wake Words Dataset)
- Image Classification (Datasets: Cifar10, Mnist)
These models represent a diverse range of architectures and data modalities. The paper does not introduce new datasets but uses these established benchmarks for evaluation.
Key Insights and Ideas
- Power Consumption as a Fingerprint: The fundamental insight is that even subtle changes in a neural network's parameters can manifest as detectable variations in the device's power consumption profile. This provides a non-intrusive way to monitor model integrity.
- Black-Box Viability: Michscan's ability to work without model parameters or owner cooperation is a significant breakthrough for real-world deployment, especially in licensed model scenarios.
- Noise Resilience through Filtering: The paper demonstrates that raw power traces are often too noisy for reliable analysis. The proposed frequency-domain filtering technique is crucial for isolating the signal of interest and enabling differentiation between benign and malicious states.
- Statistical Rigor with Mann-Whitney U-Test: Using a statistical hypothesis test like the Mann-Whitney U-Test provides a mathematically robust and quantifiable measure of confidence in detecting integrity violations, allowing for tunable sensitivity.
- Propagation of Malicious Effects: The experiments show that integrity violations, even if introduced in early layers, propagate and amplify through the network, making them detectable even when analyzing later layers.
Key Results
- 100% Detection Rate: Michscan successfully detected all integrity violations across all tested models and attack types (Trojan, poisoning, fault injection) with a P-value threshold of Pth < 10⁻⁵%.
- No False Positives/Negatives: For test similarity sizes (NRA) of 5 or 10, Michscan achieved zero false positives (benign models flagged as malicious) and zero false negatives (malicious models missed) across 3200 evaluated scenarios.
- Efficiency: Detection was achieved with as few as 5 runtime inferences, demonstrating high efficiency and minimal performance overhead.
- Severity Detection: The P-values generated by Michscan correlated with the severity of the integrity violation, with more significant modifications leading to lower P-values.
- Generalizability: The method proved effective across diverse TinyML models and attack types, indicating strong generalizability.
Limitations and Future Work
- Hardware Dependency: Power consumption patterns are hardware-specific. The pre-deployment stage (template generation and filter design) must be performed for each unique device.
- Anomaly Source Ambiguity: While Michscan reliably detects deviations in power consumption, it cannot definitively pinpoint the source of the anomaly. It could be a model integrity violation, but also a hardware failure or environmental change. The authors suggest that in security-critical applications, any significant deviation warrants investigation.
- Future Work: The authors suggest exploring the implementation of Michscan on cloud-based MLaaS platforms if voltage/power sensors with sufficient granularity become available. Further research could also focus on more precisely distinguishing between malicious modifications and other system anomalies.
Summary updated