Modern artificial intelligence faces a fundamental tension between privacy and performance. To protect sensitive data or proprietary model weights, researchers use Trusted Execution Environments (TEEs)—secure hardware enclaves like Intel SGX or AMD SEV that isolate computation from a potentially compromised operating system. However, these enclaves have historically been "CPU-only islands."
Because modern AI workloads rely on the massive parallel throughput of GPUs and other accelerators, this isolation creates a bottleneck. A researcher must choose between the robust confidentiality of a CPU enclave and the high-speed execution of an insecure GPU. Recent research aims to bridge this gap by creating Heterogeneous TEEs, extending the "circle of trust" to include high-performance accelerators.
The CPU-Accelerator Trust Gap
The core problem is that standard TEEs do not natively trust the PCIe bus or the memory resident on external cards. When a CPU enclave needs to offload work to a GPU, the data must typically be decrypted, sent over an untrusted bus, and processed in an environment where the GPU's memory is visible to the host.
As Ben Dong observes in Evaluating the Performance of the DeepSeek Model in Confidential Computing Environment, this creates a massive performance disparity. Running large-scale models like DeepSeek-R1 inside a CPU-based enclave can be orders of magnitude slower than standard GPU inference. This gap is not just a matter of speed; it is a barrier to the practical deployment of confidential AI.
Architecting a Secure Pipeline
To solve this, researchers have proposed several mechanisms to bring accelerators into the secure domain without requiring a total overhaul of commercial hardware.
One approach, pioneered by Jianping Zhu and Rui Hou in their work on HETEE, uses a trusted security controller and a programmable PCIe switch. This setup dynamically isolates off-the-shelf accelerators, allowing them to be managed as part of the secure environment. By intercepting communication at the hardware level, HETEE enables secure deep learning with minimal overhead.
On mobile and embedded devices, the challenge is even more acute due to unified memory architectures. Chenxu Wang and Kevin Leach addressed this with STRONGBOX, a lightweight TEE for Arm-based GPUs. As detailed in Building a Lightweight Trusted Execution Environment for Arm GPUs, STRONGBOX leverages existing features like Arm TrustZone and Secure Virtualization to create an isolated space for GPU tasks, achieving security with an overhead as low as 4.7%.
Hybrid and Partitioned Execution
When full hardware integration isn't possible, researchers use software-driven partitioning to balance security and speed.
- Task Partitioning: In Serdab, Tarek Elgamal and Klara Nahrstedt propose splitting neural network computation. Sensitive layers remain in a secure CPU enclave (like Intel SGX), while non-sensitive, compute-heavy layers are offloaded to untrusted accelerators.
- Shadow Tasks: The CAGE framework extends Arm’s Confidential Computing Architecture (CCA) to GPUs using a "shadow task" mechanism. This allows the system to protect data during GPU acceleration with a negligible 2.45% performance hit.
From Enclaves to Ecosystems
The ultimate goal of this research is to enable new business models for AI. For example, Lucjan Hanzlik and colleagues introduced MLCapsule, which uses enclaves to allow "guarded" offline deployment of models. This ensures a user can run a proprietary model locally without being able to steal the weights.
Similarly, Siyuan Xia and the team behind Data Station use TEEs as a "data escrow." Organizations can pool sensitive datasets for joint training without any party seeing the raw data.
As Zerui Cheng and Andrew Miller argue in OML: Open, Monetizable, and Loyal AI, the future of decentralized AI depends on "AI-native cryptography"—a combination of blockchain and secure hardware that ensures models remain loyal to their creators' constraints while running on high-performance, heterogeneous hardware.