Abstract
Compared to traditional neural networks with a single output channel, a multi-exit network has multiple exits that allow for early outputs from the model's intermediate layers, thus significantly improving computational efficiency while maintaining similar main task accuracy. Existing model stealing attacks can only steal the model's utility while failing to capture its output strategy, i.e., a set of thresholds used to determine from which exit to output. This leads to a significant decrease in computational efficiency for the extracted model, thereby losing the advantage of multi-exit networks. In this paper, we propose the first model stealing attack against multi-exit networks to extract both the model utility and the output strategy. We employ Kernel Density Estimation to analyze the target model's output strategy and use performance loss and strategy loss to guide the training of the extracted model. Furthermore, we design a novel output strategy search algorithm to maximize the consistency between the victim model and the extracted model's output behaviors. In experiments across multiple multi-exit networks and benchmark datasets, our method always achieves accuracy and efficiency closest to the victim models.
Introduction
Traditional deep learning models function like a fixed-length assembly line: every input, regardless of its complexity, must pass through every layer of the network. Multi-Exit Networks (MENs) revolutionize this approach by introducing "off-ramps" or intermediate exits. If the model achieves high confidence at an early stage, it can output a result immediately, significantly reducing computational overhead and energy consumption.
However, this efficiency introduces a new security vulnerability. While standard model stealing attacks focus on replicating the final classification logic (utility), they fail to capture the "output strategy"—the specific thresholds that determine when to exit early. Without this strategy, a stolen model becomes a "slow clone" that, while accurate, lacks the speed advantages of the original victim model. This paper introduces the first attack designed to steal both the accuracy and the efficiency of MENs.
Methodology
The proposed attack utilizes a three-stage pipeline to replicate the victim model's behavior:
- Estimation Stage (Timing Side-Channel): Because black-box APIs do not disclose which exit was used for a specific query, the researchers use execution time as a side-channel. By applying Kernel Density Estimation (KDE) to the distribution of response times, they can identify temporal boundaries between different exits and label query samples accordingly.
- Training Stage (Dual-Loss Optimization): The substitute model is trained using two distinct loss functions: a Performance Loss ($L_P$) to match classification outputs and a Strategy Loss ($L_S$) to force the clone to mimic the victim's exit decisions.
- Output Strategy Search: Following training, a specialized search algorithm identifies the optimal confidence thresholds to maximize the consistency (Closeness) between the clone and the victim.
Experimental Results
The attack was evaluated using standard benchmarks including CIFAR10, GTSRB, SVHN, and FashionMNIST. The researchers compared their method against three popular baselines: KnockOff, Noise, and ESA.
As shown in the experimental data, the proposed method significantly outperforms baselines in maintaining efficiency. While baseline attacks often produced clones that were 40% to 80% slower than the victim, this method's clones remained within 4-15% of the victim's computational cost. Furthermore, the "Closeness" (CLO) metric—measuring how often the clone and victim make the exact same exit decision—was significantly higher for the proposed method, often exceeding 60%. Notably, the KDE-based estimation stage predicted the correct exit number with nearly 100% accuracy across all tested architectures.
Conclusion
This study demonstrates that the efficiency of Multi-Exit Networks is a stealable asset. By leveraging timing side-channels and dual-loss training, attackers can create high-fidelity clones that match both the utility and the strategy of the victim. These findings suggest that future MEN deployments must consider defenses such as adversarial timing jitter or output obfuscation to protect their intellectual property and operational advantages.
Summary updated