CLiS: Plug-and-Play Attention Acceleration for Video Diffusion on TPUs via Certified Stabilization

Elisa Tsai, Shao-yu Huang, Daniel Mandragona, Wei-Yu Lin, WenXin Dong, Jevin Jiang, Chandra Devarakonda, Mehdy Bohlool, Yixin ShiarXiv 2026 · arXiv

TL;DR

CLiS (Certified Loop-invariant Stabilization) is a training-free, plug-and-play optimization that significantly speeds up video diffusion models on TPUs. By replacing the costly, repeated numerical stabilization steps in standard attention with a "certified" fixed stabilizer, it achieves up to a 19.6% reduction in end-to-end generation latency without any loss in video quality.

What Problem Does This Paper Address?

High-resolution video generation relies on Diffusion Transformers (DiTs) that process incredibly long sequences—often over 100,000 tokens for just a few seconds of video. In these models, the attention mechanism is the primary bottleneck.

While researchers usually focus on speeding up matrix multiplications (matmuls), this paper identifies an emerging hardware bottleneck on modern TPUs: the softmax operation. Specifically, "online softmax" (a standard technique to keep numbers from exploding during computation) requires repeated maximum reductions and rescaling within the inner loop of the attention kernel. On TPU architectures, these operations are surprisingly expensive because they compete for limited vector registers and have lower throughput than matmuls. As sequence lengths grow, this "non-matmul" overhead dominates the total cycle cost.

Figure 2 shows that on TPU7x, the cycle cost of softmax operations (vector and exponential units) can actually exceed that of the matrix units for common head dimensions.

Methods

The authors propose CLiS, which moves the stabilization work out of the performance-critical inner loop.

  1. Loop-Invariant Stabilization: Instead of finding the maximum value in every iteration (online softmax), CLiS calculates a fixed "stabilizer" for each row before entering the main loop. This removes the need for repeated max-reductions and rescaling.
  2. Numerical Certification: Using the Cauchy-Schwarz inequality and basic statistics of the queries and keys, CLiS derives a safe upper bound for the attention scores. It then checks if this fixed stabilizer will keep the resulting exponentials within the hardware's representable range (preventing overflow to infinity or underflow to zero).
  3. Safe Fallback: If a specific attention head fails this "certification" check, the system automatically falls back to standard, robust online softmax. This ensures that the optimization never breaks the model's numerical integrity.
  4. Implementation: CLiS is implemented using Pallas (a JAX-based kernel language for TPUs) and integrates into existing fused-attention kernels without requiring any changes to model weights or architecture.

Datasets

The researchers evaluated CLiS across several domains:

  • Video Diffusion: Tested on state-of-the-art models Wan 2.2 and LTX-2 at various resolutions (480p to 1440p) and sequence lengths (17K to 184K tokens).
  • Video Quality: Evaluated using the VBench suite to ensure no degradation in visual quality.
  • LLM Training: Preliminary experiments were conducted on causal LLMs using datasets like C4, ClimbMix, FineWeb-Edu, UltraChat, and PG-19.

Key Insights and Ideas

  • Softmax is the new bottleneck: As hardware accelerators make matmuls faster, the "simple" parts of attention like exponentiation and reduction are becoming the new critical path.
  • Register Pressure Matters: A major source of CLiS's speedup comes from reducing "spill/fill" traffic—the costly moving of data between fast registers and slower memory. By simplifying the inner loop, CLiS frees up registers, roughly halving this memory traffic.
  • Certification vs. Heuristics: Unlike prior GPU-focused methods that use empirical "best guesses" for stabilization, CLiS uses a formal proof of safety. This makes it reliable enough for both inference and training.
  • Training-Free Acceleration: Because it preserves the exact arithmetic of attention (when certified), it can be dropped into pre-trained models without any fine-tuning.
Figure 6 demonstrates how CLiS consistently reduces spill/fill memory traffic across different sequence lengths and tile sizes.

Key Results

  • Inference Speedup: CLiS reduced attention kernel latency by up to 24.0% and overall DiT-step latency by up to 19.6% on TPU v6e and TPU7x.
  • Zero Quality Loss: VBench scores confirmed that videos generated with CLiS are visually indistinguishable from those generated with standard softmax.
  • Training Gains: In LLM experiments, CLiS improved training throughput by up to 13.5% for long-context (256K) models, suggesting its utility extends far beyond just video diffusion.
  • Robustness: In a 40-step video generation run, over 98% of attention heads were successfully certified, with the remaining few safely handled by the fallback path.
Figure 1. CLiS accelerates long-sequence video diffusion attention without visually altering generation quality. Left: speedup on TPU v6e. Right: frames 0, 20, 40, 60, 80, 100 from a 720p, 7.5s (16fps, 40-step) video generated by WAN 2.2 (Team Wan, 2025).

Limitations and Future Work

  • Certification Failures: While rare, some heads do fail certification. The authors note that models without "QK-Norm" (a specific architectural feature) can see certification rates drop during early stages of training from random initialization.
  • Hardware Specificity: The current implementation is highly optimized for TPU architectures. While the mathematical principles are general, the specific performance gains from reducing register pressure are most pronounced on TPUs.
  • Future Directions: The authors suggest exploring more aggressive stabilization for the few heads that currently fail certification and extending the method to other non-matmul bottlenecks in transformer architectures.

Summary updated

Abstract

High-resolution video diffusion relies on attention over long token sequences, making attention efficiency critical to generation latency. Our analysis identifies repeated maximum reductions and accumulator rescaling in online softmax as a bottleneck in long-context TPU attention. We propose CLiS (Certified Loop-invariant Stabilization), a training-free, plug-and-play optimization for video diffusion inference. CLiS fixes per-row stabilizers before the KV loop and certifies numerical safety per head, falling back to online softmax when certification fails. It integrates into existing fused-attention kernels without changing model weights or architecture. On Wan 2.2 and LTX-2 across TPU v6e and TPU7x, CLiS reduces diffusion transformer (DiT) step latency by up to 19.6% relative to tuned online softmax without measurable quality loss. Controlled TPU adaptations of prior GPU stabilization methods show performance or numerical-safety limitations. Preliminary causal LLM experiments also show up to 13.5% higher training throughput in continued pretraining and 7.7% in supervised fine-tuning.
BibTeX
@misc{tsai2026clis,
  author = {Elisa Tsai and Shao-yu Huang and Daniel Mandragona and Wei-Yu Lin and WenXin Dong and Jevin Jiang and Chandra Devarakonda and Mehdy Bohlool and Yixin Shi},
  title = {{CLiS: Plug-and-Play Attention Acceleration for Video Diffusion on TPUs via Certified Stabilization}},
  year = {2026},
  eprint = {2611.00444},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00444},
}
Figure