ParaAnya: Accelerating Parallel Diffusion Sampling with Plug-and-Play Output Caching

Chee-En Yu, Xiao-Xi Tan, Yi-Cheng Lin, Yun-Shao Tsai, Chee-An Yu, Hung-yi LeearXiv 2026 · arXiv

TL;DR

ParaAnya is a "plug-and-play" output caching mechanism designed to accelerate parallel diffusion sampling without requiring model retraining. By reusing previously computed network outputs when latent states change minimally across iterations, it achieves up to a 5.62× speedup over serial sampling and reduces the number of function evaluations (NFE) by up to 70.1% while maintaining high image quality.

What Problem Does This Paper Address?

Diffusion models are the current state-of-the-art for generative tasks, but they suffer from a major bottleneck: sequential sampling. Generating an image requires hundreds of denoising steps, where each step depends on the previous one.

Recent "Parallel-in-Time" (PinT) methods attempt to break this chain by parallelizing work across a sliding window of timesteps. However, these methods are often inefficient. Because they iteratively refine latent states within an overlapping window, they end up evaluating the same timesteps repeatedly. The authors discovered that nearly 40% of these evaluations are redundant, involving latent inputs that have barely changed since the last iteration. ParaAnya aims to eliminate this wasted computation.

Methods

ParaAnya introduces a simple but effective caching layer between the sampling algorithm and the diffusion model (the UNet).

  1. The Cache: It maintains a mapping $C$ that associates each timestep $t$ with its most recent input latent $\hat{x}_t$ and the corresponding network output $\hat{\epsilon}_t$.
  2. Hit/Miss Logic: Before calling the network for a batch of timesteps in the current sliding window, ParaAnya calculates the mean absolute deviation between the current latent $x_t^k$ and the cached version $\hat{x}_t$.
  3. Thresholding: If the change is below a predefined threshold ($\tau_{cache}$), it’s a cache hit. The system reuses the stored output $\hat{\epsilon}_t$ instead of running the expensive model. If it exceeds the threshold, it’s a cache miss, and the network is evaluated normally.
  4. Batched Execution: Only the "misses" are dispatched to GPU workers for evaluation. The results are then merged with the "hits" to provide the full set of drift values needed for the sampler's update rule.
The ParaAnya caching mechanism identifies hits and misses within the sliding window to avoid redundant GPU calls.

This approach is algorithm-agnostic, meaning it can be dropped into various PinT frameworks like ParaDiGMS, ParaSolver, ParaTAA, and SRDS without changing their underlying logic.

Datasets

The researchers evaluated ParaAnya using Stable Diffusion v1.5 on the COCO 2017 validation set. They used 1,000 shuffled prompts to compare performance across different sampling configurations. Evaluation metrics included the Number of Function Evaluations (NFE), wall-clock speedup, and CLIP scores to ensure image-text alignment quality was preserved.

Key Insights and Ideas

  • Redundancy is an Opportunity: The iterative nature of PinT methods means that as a trajectory converges, the inputs to the model at specific timesteps become increasingly stable. ParaAnya exploits this "convergence-induced redundancy."
  • Per-Coordinate Normalization: By dividing the latent change by the dimensionality $D$, the cache threshold $\tau_{cache}$ remains robust across different image resolutions.
  • System-Level Efficiency: Unlike methods that cache across adjacent timesteps (like DeepCache), ParaAnya caches across iterations at the same timestep. This preserves the mathematical structure of the original sampling algorithm's update rules.
  • Hardware Viability: Parallel methods often incur high overhead on small GPU clusters, sometimes performing worse than serial sampling. ParaAnya significantly reduces this overhead, making parallel sampling beneficial even on a single GPU.

Key Results

ParaAnya consistently outperformed uncached baselines across all tested scenarios:

  • Speedup: It achieved a 1.30–2.43× speedup over uncached parallel samplers. When compared to standard single-GPU serial sampling, the speedup reached up to 5.62× on an 8-GPU setup.
  • NFE Reduction: The number of model evaluations was slashed by up to 70.1%.
  • Quality Preservation: CLIP scores remained nearly identical to the baselines, proving that the caching approximation does not degrade the final output quality.
  • GPU Scaling: As shown in the table below, ParaAnya makes parallel methods much more efficient. For instance, ParaSolver on a single GPU originally achieved only a 0.41× "speedup" (a significant slowdown), but with ParaAnya, it reached 1.02×, effectively neutralizing the parallelization overhead.
Table 4. Speedup vs. the number of NVIDIA Tesla V100-SXM2 32GB GPUs for DDIM (T =50, P =16) on 1,000 COCO2017 cap- tions. Baseline is 1-GPU serial generation (2.658 s/img). ParaAnya results that outperform the corresponding base method are in bold.

Limitations and Future Work

  • Fixed Threshold: The current implementation uses a fixed cache tolerance ($\tau_{cache}$). The authors suggest that online adaptation of this threshold could further optimize the trade-off between speed and accuracy.
  • Scope: While highly effective for image generation, the method has yet to be extensively tested on other modalities like video or audio diffusion, which have even longer temporal trajectories.
  • Solver Complexity: Future work could explore integrating ParaAnya with higher-order ODE solvers to see if the caching benefits persist with more complex update rules.

Summary updated

Abstract

Diffusion models have achieved remarkable success in generative tasks, but their inherently sequential sampling process introduces a severe computational bottleneck. Recent Parallel-in-Time (PinT) solvers attempt to mitigate this by parallelizing generation across a sliding window of timesteps, advancing the window only when step-wise changes stabilize. However, this overlapping window mechanism forces the network to repeatedly evaluate the same timesteps. When the input variations between iterations are minimal, these redundant evaluations lead to significant computational waste. To address this inefficiency, we propose ParaAnya, an output cache mechanism agnostic to the parallel sampling algorithm that can reduce the number of function evaluations (NFE). ParaAnya caches input-output pairs of diffusion models and reuses the cached output at overlapping timesteps. By dispatching only cache-miss timesteps to GPU workers, our approach eliminates redundant computation while preserving the structure of the underlying algorithms' update rules. We integrate ParaAnya into four representative parallel sampling algorithms and evaluate its performance on Stable Diffusion v1.5. Across four parallel samplers evaluated with DDIM on eight GPUs, ParaAnya provides $1.30$--$2.43\times$ speedups over their uncached counterparts and reduces NFE by up to 70.1\%, reaching up to a $5.62\times$ speedup over single-GPU serial sampling while maintaining comparable CLIP scores.
BibTeX
@misc{yu2026paraanya,
  author = {Chee-En Yu and Xiao-Xi Tan and Yi-Cheng Lin and Yun-Shao Tsai and Chee-An Yu and Hung-yi Lee},
  title = {{ParaAnya: Accelerating Parallel Diffusion Sampling with Plug-and-Play Output Caching}},
  year = {2026},
  eprint = {2609.36522},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.36522},
}
Figure