ManiFusion: Unlocking the High-Throughput Generation via Superposition in Manifold Space

Minkyu Kim, Baekseung Kim, Junhoo Lee, Jangho Kim, Nojun Kwak

TL;DR

ManiFusion is a novel framework that increases diffusion model throughput by processing multiple samples simultaneously within a single denoising forward pass. By leveraging the low-dimensional nature of image manifolds and a dynamic "Trajectory-Fusion Scheduling" strategy, it achieves nearly 2x throughput gains with minimal quality loss, effectively stacking with existing acceleration methods.

What Problem Does This Paper Address?

Diffusion models are the gold standard for high-fidelity image generation, but they are notoriously slow due to their iterative nature. Most research to date has focused on reducing latency—the time it takes to generate a single image—by reducing the number of denoising steps or caching intermediate features.

However, gains in per-sample latency are hitting a point of diminishing returns. The authors of ManiFusion argue for an orthogonal approach: increasing throughput. Instead of trying to make one forward pass faster, why not make one forward pass do more work? The challenge lies in "superposition": how do you mix multiple distinct image trajectories into a single shared representation without them becoming a garbled mess of pixels?

Methods

ManiFusion introduces a backbone-agnostic framework for multi-sample generation through fused-space computation.

  1. Fusion and Unfusion: The system takes $K$ independent noise inputs and fuses them into a single representation using index-conditioned encoding and summation. This fused representation is processed by the "Body Network" (the heavy lifting part of the U-Net or Transformer). Afterward, an "Unfusion" module separates the processed features back into $K$ distinct outputs.
  2. Trajectory-Fusion Scheduling (TFS): This is the paper's secret sauce. The authors observed that the early stages of diffusion (coarse structure) are more robust to fusion than the later stages (fine detail). TFS dynamically adjusts the fusion factor $K$ throughout the sampling process—for example, processing 4 samples at once during the noisy start and gradually reducing to 1 sample for the final refinement.
  3. Architectural Flexibility: The authors demonstrate ManiFusion across three main paradigms:
  4. MF-UNet: Splitting pixel-space U-Nets at the final convolutional layer.
  5. MF-DiT: Splitting Diffusion Transformers at the final linear projection.
  6. MF-AE: Applying fusion at the representation level of a latent autoencoder.
Figure 1: Illustration of the ManiFusion framework. K × B inputs is fused into B representations, jointly processed by a shared denoising backbone, then unfused via the unfusion module into K representations per group, and finally passed through a projection layer to produce K outputs.

Datasets

The framework was evaluated across a diverse set of standard benchmarks:

  • CIFAR-10: For both unconditional and class-conditional generation.
  • CelebA & CelebA-HQ: For face generation.
  • ImageNet: To test scalability on high-resolution, complex conditional generation using Diffusion Transformers (DiT).

Key Insights and Ideas

  • The Manifold Hypothesis as a Buffer: The core "aha moment" is geometric. Because natural images live on low-dimensional manifolds ($d$) embedded in high-dimensional ambient space ($D$), there is mathematically "room" to pack multiple samples ($Kd$) into one representation, provided $D \ge Kd$.
  • Throughput is Orthogonal to Latency: Because ManiFusion changes how many samples are processed rather than how they are processed, it can be combined with other methods. You can use ManiFusion alongside step-reduction (like DDIM) and feature caching (like DeepCache) to reach throughput regimes that were previously impossible.
  • Conditional Diversity Helps: Interestingly, the authors found that mixing different classes within a fused group actually makes it easier for the model to separate them during unfusion, as the distinct class signals act as anchors for the de-multiplexing process.

Key Results

ManiFusion consistently achieves nearly 2x throughput (images per second) across various backbones and samplers with negligible impact on image quality metrics like FID (Fréchet Inception Distance).

Quantitative comparison showing ManiFusion achieving nearly double the throughput of vanilla models with similar FID scores.

As shown in Figure 2, ManiFusion significantly shifts the Pareto front of the quality-throughput trade-off. In high-throughput regimes (e.g., 500+ img/s on CIFAR-10), ManiFusion maintains much lower FID scores compared to vanilla models pushed to the same speed via step reduction alone.

Plots visualizing the superior FID and sFID performance of ManiFusion at high throughput levels compared to vanilla baselines.

Limitations and Future Work

  • Training Requirement: Unlike some "plug-and-play" acceleration methods, ManiFusion currently requires full pretraining to teach the fusion/unfusion modules how to handle the shared space. This makes it computationally expensive to adapt to existing massive models.
  • Scaling Limits: While $K=2$ and $K=4$ work well, extreme fusion factors (like $K=8$) lead to noticeable quality degradation, suggesting there is a hard limit to how much information can be "superposed" before the manifold becomes too crowded.
  • Foundation Models: The study focuses on research-scale models (DiT-S, LDM). Testing this on massive text-to-image foundation models (like Stable Diffusion 3) remains an open and exciting challenge.

Summary updated

Abstract

Diffusion models achieve high-quality image generation but remain computationally expensive due to iterative denoising across many timesteps. Existing acceleration methods mainly reduce per-sample latency, but gains along this axis are becoming saturated. We instead explore an orthogonal throughput-oriented direction by processing multiple samples within a single denoising evaluation. To this end, we propose ManiFusion, a backbone-agnostic framework for multi-sample generation through fused-space computation. ManiFusion jointly processes multiple samples by fusing them before the denoising backbone and separating them afterward through unfusion. We provide a geometric perspective on how multiple samples can share a fused representation while retaining the information needed to guide their individual generation trajectories. To better align fusion allocation with denoising stages, ManiFusion uses Trajectory-Fusion Scheduling (TFS), which dynamically adjusts fusion factors throughout sampling. ManiFusion generalizes across diffusion families, including both U-Net and transformer backbones in pixel and latent spaces. We further show that ManiFusion composes with existing acceleration methods and reaches throughput regimes unattainable by prior approaches with minimal quality degradation.
BibTeX
@misc{kim2026manifusion,
  author = {Minkyu Kim and Baekseung Kim and Junhoo Lee and Jangho Kim and Nojun Kwak},
  title = {{ManiFusion: Unlocking the High-Throughput Generation via Superposition in Manifold Space}},
  year = {2026},
  eprint = {2610.21433},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.21433},
}
Figure