Knockoff Branch: Model Stealing Attack via Adding Neurons in the Pre-Trained Model

Li-Ying Hung, Cooper Cheng-Yuan KuWACV 2025

TL;DR

This paper introduces "Knockoff Branch," a novel model stealing attack that efficiently creates a substitute model by adding a small "branch" of new neurons to a frozen, pre-trained model. This approach significantly reduces the number of trainable parameters and training time compared to existing methods, while achieving competitive accuracy and even outperforming the victim model in some cases.

What Problem Does This Paper Address?

Machine learning models, especially those offered as a service (MLaaS), are vulnerable to "model stealing attacks." Attackers query these models with data to obtain input-output pairs, then train a new "knockoff" model to mimic the victim's behavior. The goal is to create a low-cost substitute for the original model, which can then be used for further malicious purposes. While previous research has focused on making these attacks more efficient in terms of query budget and improving the transferability between victim and knockoff models, there's been limited understanding of the knockoff itself – how it learns, and how to optimize its creation. This paper aims to fill this gap by investigating how to build a high-quality knockoff with minimal resources, specifically by leveraging pre-trained models.

Methods

The core innovation is the "Knockoff Branch" architecture. Instead of fine-tuning an entire pre-trained model, the authors propose freezing the pre-trained model and adding a small "branch" of new neurons. This branch is trained to learn the stolen features, effectively acting as a lightweight adapter.

The authors explore two main types of branches:

  • Model-level branch: A conceptual idea where a new component is added to the model.
  • Layer-level branch: The practical implementation, inspired by LoRA (Low-Rank Adaptation). This involves adding small, trainable "knockoff layers" to specific layers of the pre-trained model, particularly the multi-head self-attention layers in Vision Transformers (ViTs).

A key concept introduced is the "stealing rate" ($\alpha$), a hyperparameter that controls how much the knockoff branch's learned features influence the output. This rate is used both during training and testing to analyze knockoff quality and optimize performance. The authors also introduce a regularization loss (Dice loss) to ensure consistent attention weights across different stealing rates, helping to mitigate overfitting.

Datasets

  • Victim Models: The authors used pre-trained ResNet-34 and ViT models on datasets like Caltech256, CIFAR10, CIFAR100, and Indoor67.
  • Adversary Dataset: The ILSVRC-2012 ImageNet dataset was used as a data pool for generating a transfer set. A random sampling strategy was employed to query the victim APIs.

Key Insights and Ideas

  • Pre-trained models are powerful for stealing: The research demonstrates that a significant portion of a model's functionality can be stolen by only training a small branch on top of a frozen pre-trained model, suggesting that pre-trained models already encode a lot of useful knowledge.
  • Minimal resources, maximum impact: The Knockoff Branch approach drastically reduces the number of trainable parameters (down to ~0.5% of the original model) and training epochs, making model stealing more accessible and efficient.
  • Stealing rate is crucial: The "stealing rate" ($\alpha$) is not just a tuning knob; it's a critical factor influencing knockoff quality. Lower stealing rates at test time can lead to better performance, especially when dealing with out-of-distribution (OOD) data, suggesting that OOD data can cause overfitting in knockoffs.
  • Class overlap matters: The success of the method on datasets like CIFAR10 and Caltech256, which have high class overlap with ImageNet (the pre-training dataset), highlights how pre-existing knowledge in the frozen layers can significantly aid the stealing process.
  • Attention is key: Applying knockoff layers to self-attention mechanisms in transformers is particularly effective, allowing the knockoff to learn nuanced feature representations.

Key Results

  • High Accuracy: The Knockoff Branch achieved competitive accuracy, often matching or exceeding baselines like KnockoffNets and Transformer-B, despite using significantly fewer trainable parameters (hundreds of times fewer).
  • Outperforming Victim: In some scenarios, the knockoff model achieved higher accuracy than the victim model itself, particularly on datasets with high class overlap with the pre-training data.
  • Efficiency: The method requires a fraction of the trainable parameters and training epochs compared to full fine-tuning or other established methods. For example, a ViT-B knockoff required only 0.3M parameters and 30 epochs.
  • OOD Mitigation: The proposed regularization techniques and careful selection of the stealing rate helped mitigate performance degradation when using out-of-distribution transfer datasets.

Limitations and Future Work

  • Dataset Dependency: The study primarily used models trained on ImageNet. While they explored training a victim model from scratch and found performance degradation wasn't significant, more validation across diverse pre-training datasets is needed.
  • Practical Stealing Rate Determination: While the stealing rate is crucial, finding the optimal rate empirically can be challenging in a real-world black-box scenario where the victim's secret test data is unavailable. The authors propose a conservative empirical strategy of setting the stealing rate to 0.5.
  • Transformer Architecture: The success is particularly noted for transformer architectures, and further investigation into how skip connections and other architectural components influence the knockoff process is suggested.
  • Defense Strategies: The authors hope their work will contribute to a better understanding of model stealing, aiding in the design of future defense mechanisms.

Summary updated

BibTeX
@inproceedings{hung2025knockoff,
  author = {Li-Ying Hung and Cooper Cheng-Yuan Ku},
  title = {{Knockoff Branch: Model Stealing Attack via Adding Neurons in the Pre-Trained Model}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure