Parallel Sampling of Diffusion Models on SO(3)

Yan-Ting Chen, Hao-Wei Chen, Tsu-Ching Hsiao, Chun-Yi Lee

Abstract

Diffusion models are highly effective for 3D object pose estimation but suffer from slow, sequential sampling times. This paper introduces a novel algorithm that adapts Picard iteration to the $SO(3)$ manifold (3D rotations), allowing the denoising steps to be computed in parallel. The result is a nearly 5$\times$ speedup in sampling latency without any loss in accuracy or the need to retrain the underlying model.

Introduction

In 6D object pose estimation, handling "pose ambiguity" is a major challenge. For example, a symmetric object like a blank coffee mug looks identical from multiple angles; a standard regression model might average these valid poses and output an incorrect, physically impossible rotation. Diffusion models solve this by generating a distribution of valid poses by reversing a noise process step-by-step.

However, this inherently sequential nature makes them notoriously slow. While techniques exist to speed up diffusion models for image generation in flat, Euclidean space, adapting these parallelization tricks to the $SO(3)$ manifold—the curved mathematical space of 3D rotations—has remained an open problem. This paper bridges that gap to enable real-time, high-quality pose estimation.

Methodology

The authors build their parallelization technique on top of the LiePoseDiffusion model. Their approach relies on a mathematical translation and a numerical solver to bypass sequential bottlenecks.

Mapping to the Lie Algebra

The forward diffusion process on the curved $SO(3)$ manifold is interpreted as a Stochastic Differential Equation (SDE) in its corresponding "Lie algebra." Because the Lie algebra locally behaves like flat Euclidean space, it allows standard calculus and numerical methods to be applied. The reverse-time SDE is then converted into a deterministic "probability flow ODE."

Picard Iteration

Instead of solving this ODE sequentially (where step $t$ must wait for step $t-1$ to finish), the authors apply Picard iteration. This fixed-point iteration method guesses the entire trajectory of the denoising process and iteratively refines it. Because the refinement at iteration $k+1$ only depends on the guesses from iteration $k$, the computations for all timesteps can be executed simultaneously across a GPU.

Architecture of the Picard iteration process on SO(3), highlighting the parallelizable region.

As shown in the architecture above, the "Parallelizable Region" allows the score function $s_\theta$ to be evaluated for multiple timesteps in parallel. This shift from sequential to parallel computation is further clarified by comparing their respective computation graphs.

Experimental Setup

The method was evaluated on the SYMSOL dataset, which consists of images of highly symmetric objects (cylinders, cones, tetrahedrons, cubes, and icosahedrons). The dataset includes textured variants that increase complexity, as the model must recognize when surface textures break geometric symmetry. The primary goal was to reduce inference latency while maintaining the accuracy of the baseline LiePoseDiffusion model.

Results and Discussion

Profiling the baseline model revealed that the image encoder accounted for only 1.4% of processing time, while the MLP-head performing the 100-step denoising took 98.6%. This confirmed that targeting the sequential denoising loop was the most effective path for acceleration.

  • Latency Reduction: For a single sample at 100 denoising steps, the parallelized algorithm achieved a 4.9$\times$ speedup, reducing inference time from 142.0 ms to 29.0 ms. When generating 10 pose samples simultaneously, the method achieved an 8.0$\times$ speedup.
  • Quality and Accuracy: The parallelized model maintained the exact same minimum angular distance error (~1.40 degrees) as the sequential version.
  • Symmetry Capture: Qualitative visualizations confirmed the model perfectly captures both continuous symmetries (forming a ring of valid poses for a cylinder) and discrete symmetries (forming clustered points for a cube).

Conclusion

This work demonstrates that Picard iteration can be effectively adapted to non-Euclidean manifolds to accelerate diffusion model sampling. By improving latency rather than just throughput, this approach is particularly suited for real-world robotics and tracking applications. A key limitation is the "Algorithm Inefficiency" (AI) inherent in Picard iteration; the total number of mathematical operations is higher than in sequential methods, meaning speedup depends on GPU parallelization capacity. Future work could explore dynamic optimization of the iteration window size or application to other complex manifolds like $SE(3)$.

Summary updated

Abstract

In this paper, we design an algorithm to accelerate the diffusion process on the $SO(3)$ manifold. The inherently sequential nature of diffusion models necessitates substantial time for denoising perturbed data. To overcome this limitation, we proposed to adapt the numerical Picard iteration for the $SO(3)$ space. We demonstrate our algorithm on an existing method that employs diffusion models to address the pose ambiguity problem. Moreover, we show that this acceleration advantage occurs without any measurable degradation in task reward. The experiments reveal that our algorithm achieves a speed-up of up to 4.9$\times$, significantly reducing the latency for generating a single sample.
BibTeX
@inproceedings{chen2025parallel,
  author = {Yan-Ting Chen and Hao-Wei Chen and Tsu-Ching Hsiao and Chun-Yi Lee},
  title = {{Parallel Sampling of Diffusion Models on SO(3)}},
  booktitle = {MVA},
  year = {2025},
  doi = {10.23919/mva65244.2025.11175104},
}
Figure