dKV Forcing: Separating Generation and Memory Representations through a Differentiable KV Cache

Zhiqi Li, Bo ZhuarXiv 2026 · arXiv

TL;DR

dKV Forcing is a novel training framework that makes the Key-Value (KV) cache in video diffusion transformers fully differentiable, allowing the model to learn specialized memory representations. By optimizing how information is written for future use, it enables consistent, high-quality video generation for up to four minutes from models trained on only five-second clips.

What Problem Does This Paper Address?

Causal video generators produce video segment by segment. To maintain consistency, they use a KV cache—a memory buffer that stores information from past frames for the current frame to reference.

However, there's a hidden mismatch: the "Key" and "Value" states a model needs to denoise the current frame are not necessarily the same states it should store to help denoise future frames. In fact, the authors found that in pretrained models, these two representations differ significantly.

The core challenge is that standard training methods (like "Self Forcing") treat the cache as a "black box" and cut off gradients there. The model is never actually taught how to write to its memory effectively. Differentiating through a standard mutable cache is a technical nightmare—it typically leads to massive memory consumption or breaks common optimization tricks like activation checkpointing, making it impossible to train on long sequences.

Methods

The authors introduce dKV Forcing, which enables exact gradient computation from future reads back to past writes in a single temporal traversal.

1. Differentiable KV Pages

Instead of a single mutable buffer that gets overwritten, dKV Forcing uses persistent differentiable pages. Each time the model writes to memory, it commits an immutable "page." Readers access these pages through "data-free handles"—placeholders in the autograd graph that maintain connectivity without pinning the actual massive KV tensors in GPU memory until they are needed.

Figure 3. Autograd graphs and backward replay under three cache execution strategies. Top: forward computation and retained graphs. Bottom: operations replayed during backward. (a) With a mutable cache, writes modify a shared tensor in place. Checkpoint replay repeats these writes, accumulating graphs until backward runs out of memory. (b) Two-pass reconstruction builds the rollout without a graph, then recomputes the entire sequence in a differentiable forward, with layer-wise replay during backward. (c) dKV Forcing commits each write to an immutable page outside replay. The graph retains handles and checkpoint boundaries, while backward reads the pages and reconstructs one block group at a time.

2. Writer-Specific Adaptation (Writer LoRA)

To allow the model to specialize its memory, the authors add a small, low-rank residual (LoRA) specifically to the "write" path. This means the model can learn a dedicated memory representation while keeping the shared backbone for generation intact.

3. Local Rematerialization

To keep the memory footprint manageable, the system uses local rematerialization. During the backward pass, it recomputes activations in small groups of blocks. This bounds the memory overhead independently of the sequence length.

4. Rollout Dropout

The authors observed a "fidelity-motion trade-off": models with perfect memory often become "too consistent," resulting in static videos. To counter this, they use rollout dropout, randomly disabling the writer residual during some training rollouts to encourage the model to maintain natural motion.

Datasets

The models were trained using:

  • VidProM: A large-scale dataset of text-to-video prompts.
  • Wan2.1-T2V-1.3B: The base causal video generator used for fine-tuning.

Evaluation was performed on:

  • VBench: For short-term (5s) quality.
  • VBench-Long: A benchmark specifically for long-horizon (60s and 240s) consistency and quality.
  • MovieGen prompts: Used for qualitative long-form generation tests.

Key Insights and Ideas

  • Memory is distinct from Generation: The representation needed for immediate denoising is fundamentally different from the one needed for long-term context. Specializing the "writer" role is crucial for long-term stability.
  • Single Traversal is Sufficient: You don't need expensive two-pass reconstruction (like in Self Gradient Forcing) to get exact gradients. By managing the cache as a set of immutable pages, you can compute gradients efficiently in one pass.
  • The Fidelity-Motion Trade-off: There is a direct competition between "remembering the background perfectly" and "letting the subject move." Rollout dropout is a simple but effective tool to navigate this Pareto front.

Key Results

dKV Forcing significantly outperforms existing methods in long-form video generation:

  • Extreme Extrapolation: Models trained on 5s clips can generate stable, consistent video for up to 240 seconds (4 minutes)—a 46x increase over the training horizon.
  • Superior Quality: On VBench-Long, it improves aesthetic quality by 6.6 points at 60s and 8.5 points at 240s compared to the previous state-of-the-art (SGF).
  • Efficiency Gains: Compared to two-pass reconstruction methods, dKV Forcing uses 28% less resident memory and has 20.8% shorter training cycles.
  • Consistency: As seen in Figure 6, dKV Forcing maintains subject, layout, and lighting across the full four minutes, whereas baseline models like CF++ lose the scene entirely within the first minute.
Figure 6. Comparison on 240 s video generation. Frames at 0–238 s, 46 times the training clip, for CF++, SGF and dKV Forcing on the MovieGen prompt “man wiping a kitchen counter” (top) and the VBench-Long standard prompt “a cat drinking water” (bottom). CF++ has lost the scene, SGF drifts, and dKV Forcing keeps subject, layout and lighting across the full four minutes with the motion intact. More prompts are in Appendix C.5.

Limitations and Future Work

  • Model Scale: The experiments focused on a 1.3B parameter model. Scaling dKV Forcing to much larger "frontier" models remains to be explored.
  • Task Variety: The current work focuses on text-to-video. The authors plan to extend this to action-conditioned generation (e.g., robotics or gaming), where precise memory is even more critical.
  • Objective Functions: While rollout dropout helps, developing more sophisticated training objectives that inherently balance motion and consistency is an open research direction.

Summary updated

Abstract

A causal video generator produces two key--value (KV) states for every segment: the KV it uses while denoising that segment, and the KV it writes into the cache for later segments to read. In a pretrained generator the two already differ substantially, later segments attend to them differently, and replacing the written KV with the generation KV degrades the continuation. We propose dKV Forcing (dKV denotes differentiable KV) to learn a dedicated memory representation from its future use: a writer-specific low-rank residual on the shared backbone separates memory writing from generation and is trained through the future reads of what it writes. Learning a write from its future use needs gradients through the cache, which the mutable cache in common use cannot provide. dKV Forcing replaces the training-time cache by persistent differentiable pages with local rematerialization and, in one temporal traversal, computes exactly the gradient that a two-pass reconstruction of the same objective computes. One traversal computes the full memory gradient at $7\%$ generator-update overhead over detached-write training, with $28\%$ less resident memory and $20.8\%$ shorter training cycles than the two-pass reconstruction. Over Self Gradient Forcing (SGF), the same objective trained with the same traversal, writer adaptation raises VBench-Long aesthetic quality by $6.6$ points at 60\,s and $8.5$ points at 240\,s with the highest subject and background consistency, and rollout dropout of the residual keeps most of this gain while restoring the motion of training without write gradients.
BibTeX
@misc{li2026dkv,
  author = {Zhiqi Li and Bo Zhu},
  title = {{dKV Forcing: Separating Generation and Memory Representations through a Differentiable KV Cache}},
  year = {2026},
  eprint = {2611.01648},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01648},
}
Figure