Composite Enclaves: Towards Disaggregated Trusted Execution

Moritz Schneider, Aritra Dhar, Ivan Puddu, Kari Kostiainen, Srdjan CapkunIACR Trans. Cryptogr. Hardw. Embed. Syst 2022 · OpenReview

TL;DR

This paper introduces "composite enclaves," a novel approach to Trusted Execution Environments (TEEs) that extends their security guarantees to specialized hardware like GPUs and FPGAs. By allowing TEEs to securely integrate with these disaggregated resources, composite enclaves unlock the potential of modern heterogeneous computing platforms while maintaining strong security and privacy.

What Problem Does This Paper Address?

Modern computing is increasingly relying on specialized hardware (GPUs, FPGAs, ASICs) for performance and efficiency, leading to disaggregated infrastructure in data centers. However, existing Trusted Execution Environments (TEEs), which provide hardware-level security guarantees, are typically confined to the CPU. This limitation prevents TEEs from protecting code and data when it's processed by these external, specialized hardware components. A key challenge is that TEEs have a fixed hardware Trusted Computing Base (TCB), meaning they can't easily incorporate new hardware. This often forces the use of untrusted software to manage peripherals, undermining the TEE's security. The paper aims to bridge this gap by enabling TEEs to securely leverage disaggregated specialized hardware.

Methods

The core innovation is the concept of composite enclaves. Unlike traditional TEEs with a static TCB, composite enclaves feature a configurable hardware and software TCB. This is achieved by:

  • Unit Enclaves: Breaking down a composite enclave into smaller "unit enclaves," which can reside on the CPU or on specialized hardware.
  • Configurable TCB: The TCB of a composite enclave is dynamically composed of the TCBs of its constituent unit enclaves and the necessary drivers/firmware for the specialized hardware.
  • Secure Communication: Unit enclaves communicate via shared memory, with access controlled by hardware memory protection mechanisms (like RISC-V's PMP).
  • Platform Awareness: New life cycle events (connect and disconnect) are introduced to manage the dynamic nature of specialized hardware and ensure security even if a component fails or is remapped.
  • Extended Attestation: The attestation mechanism is extended to verify not only the code running on the CPU but also the identity and integrity of the integrated specialized hardware.

The authors implemented a prototype on a RISC-V FPGA using the Keystone TEE framework and demonstrated its feasibility with two case studies: an FPGA with emulated peripherals and a large-scale accelerator.

Datasets

Not discussed in the paper. The paper focuses on a hardware-software co-design and evaluation using a custom prototype and existing hardware platforms.

Key Insights and Ideas

  • The TCB is the Bottleneck: The paper clearly articulates that the fixed TCB of traditional TEEs is the primary reason they can't embrace modern disaggregated hardware.
  • Flexibility is Key: By making the TCB configurable, composite enclaves can adapt to diverse and evolving hardware landscapes.
  • Shared Memory is Efficient: Leveraging shared memory for inter-enclave communication significantly reduces overhead compared to OS-mediated communication.
  • Attestation Needs to Evolve: Simply attesting to CPU code is insufficient; attestation must extend to the entire hardware composition of a secure environment.
  • Platform Awareness is Crucial: The dynamic nature of specialized hardware requires enclaves to be aware of their environment and react to changes, preventing security breaches.

Key Results

  • Feasible Integration: The prototype successfully integrated emulated peripherals (keyboard, sensors) and a GPU-scale accelerator into composite enclaves.
  • Small TCB Overhead: For the IO peripheral case study, the TCB for drivers and firmware was a mere 2.5 KLoC.
  • Low Performance Overhead: Context switches incurred only about 220 additional cycles compared to stock Keystone.
  • Accelerator Integration: The work demonstrated the ability to integrate a complex accelerator, enabling multi-tenant isolation and attestation.
  • Small Hardware Modifications: The modifications to the RISC-V core (PMP) and Keystone framework were minimal, suggesting broad applicability.

Limitations and Future Work

  • Remote Attacker Model: The analysis primarily considers a remote attacker. Local physical adversaries, who could manipulate sensors or hardware directly, are out of scope. The authors suggest future work could incorporate memory encryption and bus integrity for stronger protection against physical attacks.
  • Limited PMP Entries: The current RISC-V specification limits PMP entries, restricting the number of unit enclaves that can coexist. The authors note that MMU-based isolation could overcome this limitation.
  • Side-Channel Attacks: While the paper acknowledges side-channel attacks are a concern, it doesn't provide specific defenses, stating it's an orthogonal problem. Future work could explore applying existing side-channel mitigation techniques.
  • Interrupt Handling: The prototype currently only evaluates polling-based synchronization, not interrupt-based synchronization, which could be a future direction.
  • Large Drivers: Integrating very large drivers (e.g., for GPUs) into a minimal TCB remains a challenge, though the authors suggest stripping them down might be possible.

Summary updated

Abstract

The ever-rising computation demand is forcing the move from the CPU to heterogeneous specialized hardware, which is readily available across modern datacenters through disaggregated infrastructure. On the other hand, trusted execution environments (TEEs), one of the most promising recent developments in hardware security, can only protect code confined in the CPU, limiting TEEs’ potential and applicability to a handful of applications. We observe that the TEEs’ hardware trusted computing base (TCB) is fixed at design time, which in practice leads to using untrusted software to employ peripherals in TEEs. Based on this observation, we propose composite enclaves with a configurable hardware and software TCB, allowing enclaves access to multiple computing and IO resources. Finally, we present two case studies of composite enclaves: i) an FPGA platform based on RISC-V Keystone connected to emulated peripherals and sensors, and ii) a large-scale accelerator. These case studies showcase a flexible but small TCB (2.5 KLoC for IO peripherals and drivers), with a low-performance overhead (only around 220 additional cycles for a context switch), thus demonstrating the feasibility of our approach and showing that it can work with a wide range of specialized hardware.
BibTeX
@article{schneider2022composite,
  author = {Moritz Schneider and Aritra Dhar and Ivan Puddu and Kari Kostiainen and Srdjan Capkun},
  title = {{Composite Enclaves: Towards Disaggregated Trusted Execution}},
  journal = {IACR Trans. Cryptogr. Hardw. Embed. Syst.},
  year = {2022},
  url = {https://openreview.net/forum?id=lGoQdE8d3q},
}
Figure