TL;DR
CloakLM is a software-only defense framework designed to protect proprietary AI models from being stolen (exfiltrated) when deployed on third-party cloud GPUs. By shuffling the virtual layout of model layers and fragmenting their physical placement in High-Bandwidth Memory (HBM), it turns readable model weights into semantic gibberish for unauthorized observers while maintaining near-native inference performance.
What Problem Does This Paper Address?
When AI companies deploy massive foundation models (like LLaMA or Qwen) to Cloud Service Providers (CSPs), the model weights represent their core intellectual property and primary economic asset. While standard cloud security measures protect user data isolation, the model weights themselves remain surprisingly vulnerable. Modern machine learning frameworks (like PyTorch and vLLM) allocate weights sequentially in large, contiguous blocks of GPU memory to maximize processing throughput.
This predictable regularity is a goldmine for attackers. Recent hardware-level exploits show that passive observers can reconstruct models losslessly just by snooping on the PCIe bus (the Hermes attack) or dumping the GPU's High-Bandwidth Memory (HBM) via driver-level side channels (the TunnelS attack). Even Confidential Computing (hardware enclaves) falls short here: because encrypting HBM at rest introduces massive latency overheads, enclaves only enforce access controls, leaving weights stored as plaintext inside the physical memory cells.
Methods
CloakLM introduces a multi-tiered, software-only obfuscation strategy that breaks the structural assumptions of physical extraction attacks without modifying the logical execution of the inference stack. It operates across three distinct tiers:
- PCIe Traffic Shaping (Tier 0): To thwart bus-level snooping, CloakLM injects benign parallel read/write traffic during idle windows on the PCIe interconnect. This drastically lowers the signal-to-noise ratio for attackers trying to map data transfers during model loading.
- Inter-layer Virtual Address Shuffling (Tier 1): Instead of instantiating layers sequentially, CloakLM randomizes the order in which layers are mapped into the virtual address space during initialization. This breaks the assumption that adjacent memory addresses correspond to consecutive model layers.
- Physical HBM Page Remapping (Tier 2): Leveraging CUDA's Virtual Memory Management (VMM) APIs, CloakLM maps contiguous virtual memory chunks into highly fragmented, non-contiguous physical HBM pages. This ensures that a raw memory dump yields scrambled, unaligned bytes rather than clean tensor segments.
Datasets
The paper evaluates CloakLM using popular open-weight foundation models rather than traditional datasets, focusing on distributed inference workloads:
- LLaMA 3.1 (8B parameters)
- Qwen 3 (14B parameters) in both single-GPU and Tensor Parallel (TP2) configurations.
- Qwen 3 MoE (30B parameters, 3B active).
Key Insights and Ideas
- Exploiting Virtual-Physical Decoupling: The core "aha" moment is realizing that LLM inference is highly bandwidth-bound rather than dependent on physical page contiguity. Because kernels saturate HBM bandwidth across many parallel requests, randomizing physical page placement across HBM banks doesn't hurt performance—the hardware naturally distributes the parallel requests evenly.
- Cost Imposition over Absolute Secrecy: The authors shift the goalpost from mathematically perfect cryptographic secrecy (which is too slow for HBM) to cost imposition. If the computational effort to reconstruct a scrambled model from a memory dump exceeds the cost of training a new model from scratch, the defense succeeds.
- CUDA Graphs as a Shield: By utilizing CUDA Graphs for steady-state inference, the framework avoids exposing runtime traces of layer dispatch order via kernel launch metadata, sealing a potential side-channel leak.
Key Results
- Structural Disruption: CloakLM successfully drops the virtual-to-physical memory contiguity of models from over 90% down to less than 3%.
- Attack Mitigation: Under a standard vLLM setup, the authors successfully reconstructed an 11 GB model from an HBM dump in about 8 hours. With CloakLM enabled, physical page dispersion makes reconstruction completely infeasible within the same time budget, even for model segments as small as 64 MB.
- Zero Runtime Latency Overhead: Across all tested models (LLaMA 3.1, Qwen 3, Qwen MoE), CloakLM introduces no measurable overhead to per-iteration prefill or decode latencies.
- Minimal Setup Cost: The initialization shuffling adds a modest, one-time delay of only 1.5 to 3 seconds depending on the model configuration.
Limitations and Future Work
- Trusted Host Assumption: CloakLM assumes the host OS, hypervisor, and GPU drivers are uncompromised. If an attacker gains root access to the host software stack, they can read the virtual-to-physical page tables directly and reverse the obfuscation.
- Inference Only: The current design targets static, long-lived inference checkpoints. Extending these structural obfuscation techniques to the dynamic, rapidly evolving memory states of training workloads remains an open challenge left for future work.
Summary updated