CloakLM: Obfuscating GPU Memory Layout to Mitigate Model Ex-filtration for Serving

Kunal Jain, Seokjin Go, Divya MahajanarXiv 2026 · arXiv

TL;DR

CloakLM is a software-only defense framework designed to protect proprietary AI models from being stolen (exfiltrated) when deployed on third-party cloud GPUs. By shuffling the virtual layout of model layers and fragmenting their physical placement in High-Bandwidth Memory (HBM), it turns readable model weights into semantic gibberish for unauthorized observers while maintaining near-native inference performance.

What Problem Does This Paper Address?

When AI companies deploy massive foundation models (like LLaMA or Qwen) to Cloud Service Providers (CSPs), the model weights represent their core intellectual property and primary economic asset. While standard cloud security measures protect user data isolation, the model weights themselves remain surprisingly vulnerable. Modern machine learning frameworks (like PyTorch and vLLM) allocate weights sequentially in large, contiguous blocks of GPU memory to maximize processing throughput.

This predictable regularity is a goldmine for attackers. Recent hardware-level exploits show that passive observers can reconstruct models losslessly just by snooping on the PCIe bus (the Hermes attack) or dumping the GPU's High-Bandwidth Memory (HBM) via driver-level side channels (the TunnelS attack). Even Confidential Computing (hardware enclaves) falls short here: because encrypting HBM at rest introduces massive latency overheads, enclaves only enforce access controls, leaving weights stored as plaintext inside the physical memory cells.

Figure 1. Model lifecycle and inference-time memory exposure. During training, model parameters are transient and reside on tightly controlled infrastructure. Once deployed for inference, stable checkpoints cross the trust boundary and become persistently resident on third-party, disaggregated accelerators. Co-tenant VMs, non-participating GPUs on the same fabric, and physical bus observers can all access or intercept model weights, through direct memory channels, shared interconnects, or PCIe snooping, without participating in or disrupting the inference workload.

Methods

CloakLM introduces a multi-tiered, software-only obfuscation strategy that breaks the structural assumptions of physical extraction attacks without modifying the logical execution of the inference stack. It operates across three distinct tiers:

  1. PCIe Traffic Shaping (Tier 0): To thwart bus-level snooping, CloakLM injects benign parallel read/write traffic during idle windows on the PCIe interconnect. This drastically lowers the signal-to-noise ratio for attackers trying to map data transfers during model loading.
  2. Inter-layer Virtual Address Shuffling (Tier 1): Instead of instantiating layers sequentially, CloakLM randomizes the order in which layers are mapped into the virtual address space during initialization. This breaks the assumption that adjacent memory addresses correspond to consecutive model layers.
  3. Physical HBM Page Remapping (Tier 2): Leveraging CUDA's Virtual Memory Management (VMM) APIs, CloakLM maps contiguous virtual memory chunks into highly fragmented, non-contiguous physical HBM pages. This ensures that a raw memory dump yields scrambled, unaligned bytes rather than clean tensor segments.
CloakLM memory obfuscation mechanism

Datasets

The paper evaluates CloakLM using popular open-weight foundation models rather than traditional datasets, focusing on distributed inference workloads:

  • LLaMA 3.1 (8B parameters)
  • Qwen 3 (14B parameters) in both single-GPU and Tensor Parallel (TP2) configurations.
  • Qwen 3 MoE (30B parameters, 3B active).

Key Insights and Ideas

  • Exploiting Virtual-Physical Decoupling: The core "aha" moment is realizing that LLM inference is highly bandwidth-bound rather than dependent on physical page contiguity. Because kernels saturate HBM bandwidth across many parallel requests, randomizing physical page placement across HBM banks doesn't hurt performance—the hardware naturally distributes the parallel requests evenly.
  • Cost Imposition over Absolute Secrecy: The authors shift the goalpost from mathematically perfect cryptographic secrecy (which is too slow for HBM) to cost imposition. If the computational effort to reconstruct a scrambled model from a memory dump exceeds the cost of training a new model from scratch, the defense succeeds.
  • CUDA Graphs as a Shield: By utilizing CUDA Graphs for steady-state inference, the framework avoids exposing runtime traces of layer dispatch order via kernel launch metadata, sealing a potential side-channel leak.

Key Results

  • Structural Disruption: CloakLM successfully drops the virtual-to-physical memory contiguity of models from over 90% down to less than 3%.
  • Attack Mitigation: Under a standard vLLM setup, the authors successfully reconstructed an 11 GB model from an HBM dump in about 8 hours. With CloakLM enabled, physical page dispersion makes reconstruction completely infeasible within the same time budget, even for model segments as small as 64 MB.
  • Zero Runtime Latency Overhead: Across all tested models (LLaMA 3.1, Qwen 3, Qwen MoE), CloakLM introduces no measurable overhead to per-iteration prefill or decode latencies.
  • Minimal Setup Cost: The initialization shuffling adds a modest, one-time delay of only 1.5 to 3 seconds depending on the model configuration.

Limitations and Future Work

  • Trusted Host Assumption: CloakLM assumes the host OS, hypervisor, and GPU drivers are uncompromised. If an attacker gains root access to the host software stack, they can read the virtual-to-physical page tables directly and reverse the obfuscation.
  • Inference Only: The current design targets static, long-lived inference checkpoints. Extending these structural obfuscation techniques to the dynamic, rapidly evolving memory states of training workloads remains an open challenge left for future work.

Summary updated

Abstract

Large foundation models deployed on third-party and shared accelerator infrastructure face a practical risk of model exfiltration that existing defenses do not fully address. In common serving deployments, model providers control the VM or bare-metal serving stack but not the surrounding hardware substrate. The host to GPU interconnect, accelerator fabric, and neighboring infrastructure components remain outside the tenant's trust boundary and have been shown to be exploitable. Hermes demonstrates lossless DNN reconstruction from passive PCIe observation, while TunnelS exfiltrates HBM contents at high throughput via driver-level access without disrupting inference. Co-tenant VMs can further access memory-mapped interfaces or misconfigured RDMA regions without physical co-location. These attacks exploit a common property of ML systems: model weights are stored in large, contiguous, and repeatedly accessed memory regions, making intercepted PCIe transfers and HBM dumps rich enough to reveal model structure and parameters. We present CloakLM, a software-only memory-obfuscation framework that removes this structural regularity without changing the inference stack's logical view of memory. CloakLM combines three mechanisms: PCIe traffic shaping, inter- and intra-layer weight shuffling, and physical HBM page remapping. Authorized execution retains a valid virtual memory layout with negligible overhead, while unauthorized observers see fragmented and semantically incoherent state. CloakLM integrates with vLLM and PyTorch, requires no hardware changes, and complements confidential computing. Evaluation on distributed inference workloads using LLaMA and Qwen models shows near-native performance while significantly increasing resistance to PCIe snooping and HBM dump attacks, making inference-time model exfiltration substantially less practical.
BibTeX
@misc{jain2026cloaklm,
  author = {Kunal Jain and Seokjin Go and Divya Mahajan},
  title = {{CloakLM: Obfuscating GPU Memory Layout to Mitigate Model Ex-filtration for Serving}},
  year = {2026},
  eprint = {2606.18400},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.18400},
}

Loading related content...

Figure