TwinShield: Secure Foundation Model Execution by Unifying TEEs and Crypto-protected Accelerators

Jiaqi Xue, Yifei Zhao, Mengxin Zheng, Fan Yao, Yan Solihin, Qian LouOpenReview 2026 · Submission to ICLR 2026 · Rejected · OpenReview

TL;DR

TwinShield is a new framework that enables secure cloud-based inference for large foundation models (FMs) by protecting both user data and proprietary model weights simultaneously. By combining novel cryptographic outsourcing protocols with a pipelined system design, it offloads roughly 87% of heavy computations to untrusted GPUs, achieving up to 5.1× speedups over traditional secure enclave executions without sacrificing accuracy.

What Problem Does This Paper Address?

Deploying large-scale models like LLaMA or Qwen in the cloud presents a "dual confidentiality" dilemma. Model developers want to protect their intellectual property (the weights), while users need to keep their data (prompts, medical records, financial info) private.

Currently, we have two main approaches, both flawed:

  1. Trusted Execution Environments (TEEs): Hardware like Intel SGX creates a "secure box" (enclave) for computation. However, these enclaves have limited memory and processing power, making them painfully slow for models with billions of parameters.
  2. Secure Offloading: Previous research tried offloading heavy math to GPUs. However, these methods usually only protected either the model or the input, not both. Furthermore, the "Attention" mechanism in Transformers is notoriously difficult to offload securely because both sides of its matrix multiplications are generated at runtime, preventing the use of standard precomputed masks.

TwinShield tackles these issues by creating a system that treats the CPU-based TEE as the "root of trust" and the GPU as a powerful but untrusted worker.

Methods

The authors introduce two primary innovations:

1. OutMult (Protocol-level Outsourcing)

To protect both the model and the data during matrix multiplication, TwinShield uses a Scale-then-Permute strategy.

  • OutAttnMult: Specifically for the Attention mechanism. The TEE takes the dynamic operands (Query, Key, Value), adds random masks, scales them with secret diagonal matrices, and shuffles the rows/columns. The GPU performs the massive multiplication on this obfuscated data. The TEE then uses its secret scaling and permutation keys to "un-wrap" the result.
  • OutLinearMult: A more lightweight version for standard linear layers where weights are known in advance.

2. OutPipe (System-level Optimization)

Even with secure offloading, the communication between the CPU enclave and the GPU becomes a massive bottleneck. TwinShield observes that attention heads are independent.

  • Pipelining: Instead of a serial "Transfer -> Compute -> Return" workflow, TwinShield breaks the workload into compute blocks. While the GPU is calculating one block, the TEE is already preparing the next one and transferring it. This "overlaps" communication and computation, significantly increasing hardware utilization.

For integrity, the system uses Freivalds’ algorithm, a probabilistic method that allows the TEE to verify the GPU's matrix multiplication results in $O(n^2)$ time rather than re-calculating the $O(n^3)$ product.

Datasets

The paper evaluates TwinShield across several state-of-the-art models rather than training on new datasets:

  • LLMs: LLaMA3 (3B and 8B), Qwen3 (14B), and Phi-4 (14B).
  • Vision-Language Models (VLMs): Qwen2.5-VL (7B) and Pixtral (12B).
  • Evaluation Benchmark: The authors used the Wikitext dataset to measure Perplexity (PPL), ensuring that their fixed-point quantization didn't degrade the models' linguistic capabilities.

Key Insights and Ideas

  • The "Dual Protection" Complexity: Protecting both operands in a multiplication is exponentially harder than protecting one. TwinShield proves that you can achieve this by combining additive masking with secret permutations and scaling.
  • Attention is the Secure Offloading Killer: Because Attention involves $Q \cdot K^T$ (where both are runtime variables), traditional "mask-and-subtract" methods fail. TwinShield’s insight is to treat these as two-variable problems that require a more robust obfuscation layer.
  • Communication is the New Bottleneck: In the world of LLMs, the time spent moving data between the TEE and the GPU can outweigh the computation itself. The move from a serial workflow to a pipelined one (OutPipe) provided a 52.4% throughput boost on its own.

Key Results

  • Speedup: TwinShield achieved 3.3× to 5.1× speedups over TEE-only baselines. The larger the model and the longer the input sequence, the greater the performance gain.
  • Offloading Efficiency: The framework successfully offloads approximately 87% of all computations to the untrusted accelerator.
  • Accuracy: The transition from floating-point to 8-bit fixed-point quantization resulted in only marginal increases in perplexity (e.g., LLaMA3-8B went from 7.14 to 7.84), meaning the model remains highly effective.
  • Security Level: The authors estimate their obfuscation provides roughly 8,990 bits of security for a typical input length, far exceeding standard 128/256-bit requirements.

Limitations and Future Work

  • Side-Channel Attacks: The authors acknowledge that TEEs like Intel SGX are vulnerable to side-channel attacks (like Foreshadow). While they mention existing defenses (like constant-time implementation), they consider these "orthogonal" to their work and do not implement them directly.
  • Quantization Sensitivity: While the perplexity increase was marginal, some highly sensitive tasks might be more affected by 8-bit quantization.
  • Hardware Specifics: The current implementation is optimized for Intel SGX and NVIDIA GPUs. Adapting this to other TEEs (like ARM TrustZone) or different accelerators (TPUs, FPGAs) would require further engineering.

Summary updated

Abstract

Recent advances in Transformer-based foundation models (FMs) have driven significant developments across diverse AI tasks, facilitating their deployment in security-sensitive domains. Despite their capabilities, FMs impose substantial inference costs, driving reliance on third-party cloud infrastructure equipped with high-performance computation resources. However, these cloud platforms cannot be fully trusted and remain vulnerable to data breaches, introducing dual confidentiality challenges: protecting user data from exposure and safeguarding models against unauthorized access. Mainstream protection mechanisms leverage trusted execution environments (TEEs), where confidentiality and integrity are enforced through hardware-based isolation, encryption, and integrity verification. But executing inference entirely within TEEs incurs a significant overhead, which is further exacerbated in large-scale FMs. Recent studies have proposed schemes that combine TEEs with untrusted accelerators (e.g., GPUs) to offload partial inference operations. However, prior offloading schemes cannot solve dual confidentiality challenges in FM inference, since operations such as ***Attention*** depend on dynamic operands that prevent secure precomputation and must remain within TEEs. Moreover, the communication overhead between TEEs and accelerators grows dramatically with model scale, constituting a new system design challenge for FMs. To address these challenges, we propose ***Twinshield***, a framework that enables secure inference of Transformer-based FMs in heterogeneous TEE–accelerator systems with dual protection for both model and data. ***Twinshield*** improves efficiency through ***protocol-level*** outsourcing, which securely offloads the majority of operations to accelerators, and enhances throughput via a ***system-level*** design that overlaps TEE preparation, communication, and accelerator execution. Our evaluation on representative LLMs and VLMs shows that ***Twinshield*** offloads about 87% of computations to accelerators and achieves $3.3\times$–$5.1\times$ speedups over baselines. The code is publicly available at https://anonymous.4open.science/r/Twinshield/README.md.
BibTeX
@misc{xue2026twinshield,
  author = {Jiaqi Xue and Yifei Zhao and Mengxin Zheng and Fan Yao and Yan Solihin and Qian Lou},
  title = {{TwinShield: Secure Foundation Model Execution by Unifying TEEs and Crypto-protected Accelerators}},
  year = {2026},
  url = {https://openreview.net/forum?id=nCJF1CDapQ},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}
Figure