TL;DR
JITterFlip is a novel bit-flip attack that targets the host-side CPU control plane of JIT-compiled LLM serving rather than the model's weights or GPU computation. By using Rowhammer to flip a single bit in PyTorch's compilation stack, attackers can either force an LLM to output gibberish or trigger a stealthy "sponge attack" that multiplies latency by up to 125× while keeping the output perfectly correct, completely bypassing existing defenses.
What Problem Does This Paper Address?
Modern Large Language Models (LLMs) are widely served using Just-in-Time (JIT) compilation frameworks (such as PyTorch's torch.compile) to minimize framework overhead and optimize GPU execution. While previous hardware-based bit-flip attacks (BFAs)—like Rowhammer—have successfully targeted model weights or numerical libraries to disrupt AI inference, they suffer from two major flaws: they require deep, model-specific knowledge, and they cause obvious output corruption ("inference depletion") that semantic security filters easily catch.
JITterFlip addresses this gap by shifting the target from the computation plane (the GPU and model weights) to the control plane (the host CPU orchestrating the JIT artifacts). This exposes an entirely new, model-agnostic vulnerability surface in modern AI serving stacks.
Methods
The authors introduce a Decision-Guided Fault-Vulnerable Code Analysis framework to systematically find exploitable bits within massive JIT compiler stacks:
- Source Localization: They narrow down PyTorch's JIT stack (from 139.1K lines of code to 34.7K) by isolating code responsible for two critical decisions: compiled-artifact selection and GPU-work submission.
- Archetype-Guided Search: They define 5 structural archetypes of vulnerable conditional branches (e.g., cache-match gating, launch-path resolution). Using LLVM Intermediate Representation (IR), they perform backward predicate provenance analysis (tracing what a branch is testing) and forward successor-region analysis (evaluating the behavioral difference between branch paths) to rank the most impactful targets.
- Exploitation: They profile physical memory for Rowhammer-vulnerable bits and use memory waylaying to force the OS to load the target JIT code page onto the flippable DRAM location, executing a precise single-bit inversion at runtime.
Datasets
The method was evaluated across text-only and multimodal workloads using:
- WikiText-2: Used for evaluating text-generation perplexity (PPL) on text models.
- ScienceQA: A dataset of image-based multiple-choice questions used to evaluate multimodal LLMs.
- Models Evaluated: Qwen3-8B, DeepSeek-R1-Distill-Qwen-7B, Gemma-3-4B, and Qwen2.5-VL-7B (plus Qwen3-0.6B for defense evaluations).
Key Insights and Ideas
- CPU Faults, GPU Consequences: Even though LLM math happens on the GPU, the host CPU holds the steering wheel. Corrupting CPU-side JIT decisions seamlessly propagates across the CPU-GPU boundary to disrupt GPU-executed inference without needing direct access to GPU memory.
- The Stealthy Sponge Attack: By flipping a bit in the compiled-cache lookup logic, the attacker can force the system to repeatedly recompile or fall back to slower execution paths. The model still produces the exact right answer, making it invisible to semantic defenses, but it consumes massive computational resources, causing a severe Denial of Service (DoS).
- Defense Blindspots: Current LLM defenses (like FaR or LM-Fix) focus heavily on protecting weight integrity or checking output sanity. Because JITterFlip leaves weights untouched and can preserve output correctness, it completely bypasses these state-of-the-art protections.
Key Results
- High Precision: The automated analysis identified vulnerable branches with 25% precision for output corruption and 50% precision for latency amplification, vastly outperforming random or sequential guessing (0%).
- Gibberish Generation: Successful attacks caused massive output corruption, raising Perplexity (PPL) ratios by $15.45\times$ to over $2.48\times 10^6\times$.
- Latency Amplification: In end-to-end Rowhammer hardware attacks, the "sponge attack" successfully slowed down inference by 97.38× to 124.97× while preserving identical token outputs.
- Transferability: The identified vulnerable branches were highly transferable across different model architectures and modalities (text vs. vision-language) because they target shared framework logic rather than specific model parameters.
Limitations and Future Work
- Framework Dependency: The current implementation is tailored to PyTorch 2.9.1. Porting JITterFlip to other frameworks (like JAX/XLA) or newer PyTorch versions requires re-running the source localization and static analysis pipeline.
- Hardware Constraints: End-to-end attacks were demonstrated on a non-ECC DDR4 platform. While the authors note that recent literature shows Rowhammer can bypass ECC and DDR5 mitigations, they did not explicitly test JITterFlip on these newer hardware environments.
Summary updated