CoreGuard: Safeguarding Foundational Capabilities of LLMs Against Model Stealing in Edge Deployment

Qinfeng Li, Yangfan Xie, Tianyu Du, Zhiqiang Shen, Zhenghan Qin, Hao Peng, Xinkui Zhao, Xianwei Zhu, Jianwei Yin, Xuhong Zhang

TL;DR

This paper introduces CoreGuard, a novel method to protect large language models (LLMs) deployed on edge devices from model stealing attacks. CoreGuard uses a lightweight, permutation-based approach within a Trusted Execution Environment (TEE) to safeguard the LLM's foundational capabilities, achieving strong security with minimal overhead.

What Problem Does This Paper Address?

Large language models (LLMs) are increasingly being deployed on edge devices for tasks requiring low latency and high privacy. However, this trend introduces a significant security vulnerability: "foundational capability stealing." Attackers can potentially extract these powerful, general-purpose LLMs and misuse their inherent generalization abilities for unauthorized purposes, such as creating derivative models or performing new tasks without proper authorization. Existing defenses often struggle with the computational and communication overhead required for edge deployment, especially for complex LLMs. This paper aims to bridge this gap by providing an efficient and robust protection mechanism specifically for edge-deployed LLMs.

Methods

CoreGuard employs a two-phase approach: model locking (before deployment) and inference authorization (during inference).

  1. Model Locking: The core idea is to "lock" the LLM's linear layers by permuting their weight matrices. This permutation renders the model unusable without a corresponding authorization. The key innovation here is the propagation protocol. Instead of requiring TEE authorization for every layer, CoreGuard permutes the output of certain layers, allowing the authorization to propagate through the network. This significantly reduces the need for frequent TEE interactions.
  2. Inference Authorization: During inference, a lightweight authorization mechanism within a TEE is used. This involves a permutation of the input features, which is then applied to the model's weights. To further enhance security and efficiency, CoreGuard uses a one-time pad (OTP) encryption combined with a permutation to obfuscate the process and prevent input-output differencing. The TEE performs a single authorization step, and the authorization is then propagated through the network via the pre-defined permutations.

Datasets

The paper evaluates CoreGuard on four domain-specific tasks using the following datasets:

  • GSM8k: Mathematics problems.
  • Spider: Code generation.
  • PubMedQA: Medical question answering.
  • SQuAD: Reading comprehension.

The models used for evaluation include:

  • Qwen2-0.5B-Instruct
  • Gemma2-2B-it
  • ChatGLM3-6B-32k
  • LLaMA3-8B-Instruct

Key Insights and Ideas

  • Foundational Capability Stealing is a Real Threat: The paper highlights that the generalizability of LLMs makes them particularly vulnerable to misuse, even if only partially extracted. This is distinct from traditional task-specific model stealing.
  • Propagation Protocol is Key to Efficiency: The most significant insight is the development of a propagation protocol that allows authorization to cascade through the network. This drastically reduces the communication overhead between the TEE and the GPU, a major bottleneck in previous TEE-based defenses.
  • Permutation as a Lightweight Lock: Using row permutations on weight matrices and column permutations on inputs provides an effective and computationally inexpensive way to "lock" the model.
  • Authorization Point Matters: The paper empirically shows that placing the authorization point in the middle of the network offers the best security against model stealing attacks, as it requires attackers to recover a larger portion of the model.

Key Results

  • Strong Security: CoreGuard achieves near black-box security levels against model stealing attacks, with attack accuracies significantly lower than unprotected models and comparable to the upper bound. Unauthorized inference accuracy is reported as 0%.
  • Minimal Overhead: CoreGuard demonstrates substantially lower TEE execution and TEE-GPU transfer overheads compared to existing methods. TEE execution overhead is less than 0.00117% of the original model's FLOPS, and transfer overhead is nearly two orders of magnitude lower than PSP methods.
  • No Accuracy Loss: The protected models exhibit minimal to no degradation in accuracy compared to the original models, with fluctuations attributed to precision limitations rather than the defense mechanism itself.

Limitations and Future Work

  • Side-Channel Attacks: The paper acknowledges that CoreGuard, like other TEE-based solutions, is potentially vulnerable to side-channel attacks. However, it suggests that existing software and hardware defenses for side-channel leaks can be integrated.
  • TEE in GPUs: The current reliance on CPU-based TEEs (like ARM TrustZone) is noted as a limitation for future work, as more advanced TEEs integrated directly into GPUs are still in early stages and not widely available for edge devices.
  • Real-World Environments: While the core framework is hardware-agnostic, the paper focuses on the framework itself and does not delve into device-specific implementations. Future work could explore practical deployment on various edge devices.

Summary updated

Abstract

Proprietary large language models (LLMs) demonstrate exceptional generalization ability across various tasks. Additionally, deploying LLMs on edge devices is trending for efficiency and privacy reasons. However, edge deployment of proprietary LLMs introduces new security threats: attackers who obtain an edge-deployed LLM can easily use it as a base model for various tasks due to its high generalization ability, which we call foundational capability stealing. Unfortunately, existing model protection mechanisms are often task-specific and fail to protect general-purpose LLMs, as they mainly focus on protecting task-related parameters using trusted execution environments (TEEs). Although some recent TEE-based methods are able to protect the overall model parameters in a computation-efficient way, they still suffer from prohibitive communication costs between TEE and CPU/GPU, making it impractical to deploy for edge LLMs. To protect the foundational capabilities of edge LLMs, we propose CoreGuard, a computation- and communication-efficient model protection approach against model stealing on edge devices. The core component of CoreGuard is a lightweight and propagative authorization module residing in TEE. Extensive experiments show that CoreGuard achieves the same security protection as the black-box security guarantees with negligible overhead.
BibTeX
@inproceedings{li2025coreguard,
  author = {Qinfeng Li and Yangfan Xie and Tianyu Du and Zhiqiang Shen and Zhenghan Qin and Hao Peng and Xinkui Zhao and Xianwei Zhu and Jianwei Yin and Xuhong Zhang},
  title = {{CoreGuard: Safeguarding Foundational Capabilities of LLMs Against Model Stealing in Edge Deployment}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=84dnGT4ajt},
}
Figure