Exploiting Temporal Redundancy in Iterative Generative Models

381 concepts176 papers1 proposal

Iterative generative models—such as diffusion models, flow-matching systems, and autoregressive decoders—generate high-fidelity outputs by making small, incremental updates over many sequential steps. Standard hardware execution treats each step as an independent problem, recomputing all internal activations, attention matrices, and feed-forward layers from scratch.

However, because the underlying state evolves gradually, the internal representations across consecutive steps exhibit massive temporal redundancy. Activations change far less between adjacent time steps than they do across spatial dimensions within a single step. By caching and reusing historical computations across sequential iterations, we can bypass redundant operations, drastically reducing latency and energy consumption with negligible impact on final quality.

Amortizing the Attention Bottleneck

The most computationally expensive operation in modern Transformer-based architectures is the self-attention mechanism, which scales quadratically with sequence length. In iterative generation, the relative importance of tokens (which tokens attend to which) remains remarkably stable across consecutive steps.

Rosco Hunter et al. demonstrate in Fast Sampling Through The Reuse Of Attention Maps In Diffusion Models that high-dimensional key-query products can be cached and reused between adjacent sampling steps, eliminating redundant matrix multiplications.

This temporal stability applies not just to the raw attention weights, but also to the sparse attention masks derived from them. For instance, Sankeerth Durvasula et al. introduce a fine-grained, slice-based attention masking framework in FG-Attn: Leveraging Fine-Grained Sparsity In Diffusion Transformers that reuses importance maps across multiple denoising steps.

Similarly, in the text domain, Zeqing Wang et al. exploit this property in SparseD: Sparse Attention for Diffusion Language Models by computing a sparse attention mask once and reusing it across subsequent steps. Even in autoregressive long-context scenarios, the set of influential tokens changes slowly. Heejun Lee et al. leverage this in InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU by caching and periodically refreshing sparse attention masks rather than recomputing them at every single decoding step.

Standard Inference:
Step t:   [Compute Q, K] -> [Compute Attention Map] -> [Output]
Step t+1: [Compute Q, K] -> [Compute Attention Map] -> [Output]

Temporal Caching Inference:
Step t:   [Compute Q, K] -> [Compute Attention Map] -> [Cache Map] -> [Output]
Step t+1:                                              [Reuse Map] -> [Output]

Block-Wise Caching and the U-Shaped Trajectory

Beyond isolating attention maps, can we skip entire structural blocks within a network? The challenge lies in knowing when and where features are stable enough to bypass computation.

Recent work reveals that feature variation across denoising steps follows a predictable U-shaped trajectory. At the very beginning of the diffusion process, the model establishes global layout and macro-structures; at the very end, it refines fine-grained details. In both phases, features change rapidly. However, during the middle stages, the internal activations stabilize significantly.

Hanshuai Cui et al. exploit this phenomenon in BWCache: Accelerating Video Diffusion Transformers through Block-Wise Caching, dynamically evaluating the redundancy of individual transformer blocks and bypassing updates when the relative difference falls below a threshold. To make this process adaptive without manual tuning, Xuan Shen et al. propose LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers, which introduces lightweight layers that dynamically decide whether to reuse cached attention or feedforward outputs based on similarity thresholds.

This macro-level caching is highly effective; Joseph Liu 0001 et al. show in SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformers that caching stable representations across adjacent timesteps can yield up to a 71% inference speedup across multiple modalities.

Tracking the Trajectory via Residuals and Velocity Alignment

Instead of completely freezing a feature map or block output—which can introduce approximation errors or "stuttering" artifacts—some frameworks focus on caching the dynamics of change.

Pengtao Chen demonstrates in $\Delta$-DiT: Accelerating Diffusion Transformers without training via Denoising Property Alignment that caching the incremental differences ($\Delta$) between feature maps across blocks is highly effective, as these small deltas remain highly correlated across consecutive steps.

A similar intuition applies to the directional velocity of the generation process. In RegionE: Adaptive Region-Aware Generation for Efficient Image Editing, Pengtao Chen et al. show that the direction of change in transformer outputs remains consistent even as its magnitude decays. By applying an adaptive decay factor to previous velocity vectors, they safely skip expensive computations.

At the hardware level, this gradual shifting of values can be translated directly into energy savings. Sungbin Kim et al. design a hardware-software co-design in Ditto: Accelerating Diffusion Model via Temporal Value Similarity that processes only the small deltas between time steps using reduced bit-widths and zero-skipping, proving that temporal redundancy can be directly converted into physical power efficiency.

Cross-Modal and Non-Generative Extensions

The principle of temporal feature caching extends naturally beyond standard image and text generation into robotics, video processing, and optimization loops.

In real-time robotic control, Vision-Language-Action (VLA) models must process heavy perceptual inputs alongside action generation loops. Yuanchun Guo et al. introduce Reflex: Real-Time Vision-Language-Action Control through Streaming Inference, which decouples the perception backbone from the action-denoising loop. Because the visual context is invariant to the immediate denoising step, caching key-value states transforms a quadratic complexity update into a constant-time operation.

Even outside of generative modeling, iterative optimization algorithms display identical patterns. Zhichao Hou et al. demonstrate in Fine-Grained Iterative Adversarial Attacks with Limited Computation Budget that neural network activations across successive steps of adversarial attacks (like PGD) become highly correlated. By selectively recomputing layers only when their residual changes cross a threshold, they achieve massive computational savings under tight compute budgets.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 180 papers
  2. Version 1 · 2026-10-04 · 270 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_feature_caching_temporal_featur_6abfb682,
  title = {temporal feature caching (temporal feature redundancy)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-feature-caching-temporal-feature-redundancy/rdr_6abfb682127b41aa8712e7e4d6304ed5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_temporal_feature_caching_temporal_featur_6abfb682_v2,
  title = {temporal feature caching (temporal feature redundancy)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-feature-caching-temporal-feature-redundancy/rdr_6abfb682127b41aa8712e7e4d6304ed5/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake