TL;DR
DiffusionSSD is a hardware-software co-design framework that enables extreme visual compression (below 0.01 bits-per-pixel) on resource-constrained edge devices. By combining a lightweight "dual-stage" diffusion algorithm with a novel architecture that offloads computation directly to storage and memory, it reduces computation costs by 92.7% and more than doubles coding throughput compared to traditional NPU-based approaches.
What Problem Does This Paper Address?
We are drowning in high-resolution visual data. From 4K cameras to VR headsets generating gigabytes of content per minute, edge devices are struggling to store it all. While traditional codecs like JPEG or H.264 are fast, their quality "collapses" at extreme compression ratios (very low bitrates).
Neural codecs, particularly those based on diffusion models, have emerged as a powerful alternative, capable of maintaining high perceptual quality even when data is heavily compressed. However, they face two massive hurdles for edge deployment:
- Computational Intensity: Diffusion decoders are notoriously slow because they typically require many iterative denoising steps.
- Architectural Bottlenecks: In standard systems, neural codecs hog the NPU, interfering with other AI tasks. Furthermore, moving massive amounts of visual data between storage (Flash), memory (DRAM), and the processor (SoC) creates significant I/O overhead and drains battery life.
Methods
The authors propose DiffusionSSD, a cross-level framework that tackles these issues through a three-pronged approach:
1. Algorithm: Dual-stage Diffusion
To make diffusion practical, the authors introduce a lightweight Dual-stage Diffusion scheme. Instead of processing a high-resolution image all at once, they partition it into patches.
- Stage 1 (Local): Performs denoising on individual patches independently.
- Stage 2 (Global): A final refinement step on the merged image to ensure global coherence and fix "seams" between patches. Crucially, they use single-step diffusion initialized at a high Signal-to-Noise Ratio (SNR) state. By starting with the upsampled features from the bitstream rather than pure noise, the model skips the most time-consuming parts of the denoising process.
2. Architecture: Cross-level Near-Data Processing (NDP)
Rather than running everything on the main SoC's NPU, DiffusionSSD offloads the work to the "data side" using two specialized components integrated into a single package (uMCP):
- computational Universal Flash Storage (cUFS): The storage controller is augmented with a small NPU core and a dedicated Tensor Entropy Coding Unit (TECU) to handle the bitstream math right next to the Flash memory.
- LPDDR-PIM (Processing-in-Memory): The DRAM is equipped with SIMD-style compute units to handle the heavy lifting of the diffusion layers.
A direct link between the cUFS and PIM allows data to bypass the main SoC entirely during compression and decompression.
3. System: Patch-Workload Co-aware Scheduling
To manage this heterogeneous hardware, the authors developed a scheduling scheme that intelligently maps different parts of the neural network to either the cUFS or the PIM based on which unit is better suited for a specific operator (e.g., convolutions vs. entropy coding).
Datasets
The models were trained on the Vimeo-90K dataset. For evaluation, the authors used:
- Tecnick: A standard benchmark for image compression.
- UVG 2K: A dataset for high-resolution video compression.
Key Insights and Ideas
- The "Aha" Moment for Speed: Standard diffusion models waste time "rediscovering" structure from pure noise. By initializing the diffusion process with the structural information already present in the compressed latent features, you can achieve high quality in just one or two steps.
- Bypassing the SoC Middleman: Moving data is often more expensive than computing on it. By creating a "short-circuit" between storage and memory-side compute, DiffusionSSD eliminates the energy and latency penalties of the traditional I/O path.
- Patch-level Parallelism: Patchification isn't just for saving memory; it creates natural units for scheduling across different hardware accelerators, allowing for efficient pipelining.
Key Results
- Efficiency: Dual-stage Diffusion reduces computational costs by up to 92.7% compared to standard multi-step diffusion decoders.
- Throughput: The cUFS+PIM architecture improves coding throughput by 2.32× over a standard NPU+DRAM setup and 2.91× over existing near-data designs like SmartSSD.
- Quality: It maintains excellent perceptual quality at bitrates below 0.01 bpp, a regime where traditional JPEG and H.265 codecs produce unwatchable results.
- Energy: The collaborative execution achieves energy reductions of 17.9% for encoding and 24.6% for decoding per frame.
Limitations and Future Work
- Generative Fidelity: While perceptual metrics (like LPIPS) are strong, the Fréchet Inception Distance (FID) — a measure of how "realistic" generated images look — lags slightly behind multi-step models. This is a known trade-off when reducing denoising steps.
- Hardware Complexity: The proposed uMCP architecture requires advanced packaging (like UCIe links) which, while technically feasible, adds complexity to the manufacturing process.
- Entropy Coding Support: The dedicated TECU unit currently supports specific arithmetic coding schemes; expanding this to a wider variety of entropy models would increase the framework's flexibility.
For a deeper dive into the architecture and mathematical formulations, you can find the full paper here.
Summary updated