JITterFlip: Uncovering Fault Attack Surfaces in JIT-Compiled LLM Serving

Tairui Wang, Zhi Zhang, Yansong Gao, Xin Zhang, Qingni Shen, Zhonghai WuarXiv 2026 · arXiv

TL;DR

JITterFlip is a novel bit-flip attack that targets the host-side CPU control plane of JIT-compiled LLM serving rather than the model's weights or GPU computation. By using Rowhammer to flip a single bit in PyTorch's compilation stack, attackers can either force an LLM to output gibberish or trigger a stealthy "sponge attack" that multiplies latency by up to 125× while keeping the output perfectly correct, completely bypassing existing defenses.

What Problem Does This Paper Address?

Modern Large Language Models (LLMs) are widely served using Just-in-Time (JIT) compilation frameworks (such as PyTorch's torch.compile) to minimize framework overhead and optimize GPU execution. While previous hardware-based bit-flip attacks (BFAs)—like Rowhammer—have successfully targeted model weights or numerical libraries to disrupt AI inference, they suffer from two major flaws: they require deep, model-specific knowledge, and they cause obvious output corruption ("inference depletion") that semantic security filters easily catch.

JITterFlip addresses this gap by shifting the target from the computation plane (the GPU and model weights) to the control plane (the host CPU orchestrating the JIT artifacts). This exposes an entirely new, model-agnostic vulnerability surface in modern AI serving stacks.

Methods

The authors introduce a Decision-Guided Fault-Vulnerable Code Analysis framework to systematically find exploitable bits within massive JIT compiler stacks:

  1. Source Localization: They narrow down PyTorch's JIT stack (from 139.1K lines of code to 34.7K) by isolating code responsible for two critical decisions: compiled-artifact selection and GPU-work submission.
  2. Archetype-Guided Search: They define 5 structural archetypes of vulnerable conditional branches (e.g., cache-match gating, launch-path resolution). Using LLVM Intermediate Representation (IR), they perform backward predicate provenance analysis (tracing what a branch is testing) and forward successor-region analysis (evaluating the behavioral difference between branch paths) to rank the most impactful targets.
  3. Exploitation: They profile physical memory for Rowhammer-vulnerable bits and use memory waylaying to force the OS to load the target JIT code page onto the flippable DRAM location, executing a precise single-bit inversion at runtime.

Datasets

The method was evaluated across text-only and multimodal workloads using:

  • WikiText-2: Used for evaluating text-generation perplexity (PPL) on text models.
  • ScienceQA: A dataset of image-based multiple-choice questions used to evaluate multimodal LLMs.
  • Models Evaluated: Qwen3-8B, DeepSeek-R1-Distill-Qwen-7B, Gemma-3-4B, and Qwen2.5-VL-7B (plus Qwen3-0.6B for defense evaluations).

Key Insights and Ideas

  • CPU Faults, GPU Consequences: Even though LLM math happens on the GPU, the host CPU holds the steering wheel. Corrupting CPU-side JIT decisions seamlessly propagates across the CPU-GPU boundary to disrupt GPU-executed inference without needing direct access to GPU memory.
  • The Stealthy Sponge Attack: By flipping a bit in the compiled-cache lookup logic, the attacker can force the system to repeatedly recompile or fall back to slower execution paths. The model still produces the exact right answer, making it invisible to semantic defenses, but it consumes massive computational resources, causing a severe Denial of Service (DoS).
  • Defense Blindspots: Current LLM defenses (like FaR or LM-Fix) focus heavily on protecting weight integrity or checking output sanity. Because JITterFlip leaves weights untouched and can preserve output correctness, it completely bypasses these state-of-the-art protections.

Key Results

  • High Precision: The automated analysis identified vulnerable branches with 25% precision for output corruption and 50% precision for latency amplification, vastly outperforming random or sequential guessing (0%).
  • Gibberish Generation: Successful attacks caused massive output corruption, raising Perplexity (PPL) ratios by $15.45\times$ to over $2.48\times 10^6\times$.
  • Latency Amplification: In end-to-end Rowhammer hardware attacks, the "sponge attack" successfully slowed down inference by 97.38× to 124.97× while preserving identical token outputs.
  • Transferability: The identified vulnerable branches were highly transferable across different model architectures and modalities (text vs. vision-language) because they target shared framework logic rather than specific model parameters.

Limitations and Future Work

  • Framework Dependency: The current implementation is tailored to PyTorch 2.9.1. Porting JITterFlip to other frameworks (like JAX/XLA) or newer PyTorch versions requires re-running the source localization and static analysis pipeline.
  • Hardware Constraints: End-to-end attacks were demonstrated on a non-ECC DDR4 platform. While the authors note that recent literature shows Rowhammer can bypass ECC and DDR5 mitigations, they did not explicitly test JITterFlip on these newer hardware environments.

Summary updated

Abstract

LLMs are widely deployed through cloud-hosted inference services, where Just-in-Time (JIT) compilation is used to reduce recurring framework and GPU-launch overhead. JIT serving introduces a host-side control plane that selects compiled artifacts and orchestrates their execution on the GPU. Meanwhile, the shared cloud setting has motivated a growing body of bit-flip attacks (BFAs) against LLM/DNN inference. Most existing BFAs target model parameters or weights and require model-specific knowledge. A smaller body of work reduces this dependency by faulting executable code, yet still corrupts code that directly implements model computation, limiting their attack effect to inference depletion. We present JITterFlip, the first BFA targeting the host-side JIT serving control plane of GPU-based LLM inference. By faulting CPU-resident serving decisions rather than model computation, JITterFlip enables both gibberish output generation and a correct-output sponge attack. To identify exploitable targets in a large JIT compiler stack, JITterFlip develops a decision-guided fault-vulnerable code analysis. Across four text and multimodal LLM workloads, the identified vulnerable code faults exhibit cross-model transferability, produce gibberish outputs with PPL ratios of $15.45\times$ to $2.48{\times}10^{6}\times$, and demonstrate correct-output sponge attacks with latency amplification of $2.03\times$ to $181.90\times$. JITterFlip also bypasses recent BFA defenses for LLMs while retaining both attack effects. Last, we demonstrate end-to-end Rowhammer attacks across four LLMs: a single bit flip in CPU-resident branch code propagates across the CPU-GPU boundary to disrupt GPU-executed inference without direct access to GPU memory, reaching up to $7.23{\times}10^{6}\times$ PPL amplification or $124.97\times$ latency amplification while preserving the exact generated output.
BibTeX
@misc{wang2026jitterflip,
  author = {Tairui Wang and Zhi Zhang and Yansong Gao and Xin Zhang and Qingni Shen and Zhonghai Wu},
  title = {{JITterFlip: Uncovering Fault Attack Surfaces in JIT-Compiled LLM Serving}},
  year = {2026},
  eprint = {2608.29745},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.29745},
}
Figure