Flex-DDCM: Zero-Shot Task-Oriented Image Compression via Diffusion Trajectory Guidance

Yixin Gao, Runsen Feng, Xin Li, Zhibo ChenarXiv 2026 · arXiv

TL;DR

Flex-DDCM is a zero-shot image compression framework that uses pretrained diffusion models to flexibly balance human-centric quality and machine-centric utility. By guiding the diffusion trajectory at the encoder with task-specific gradients, it achieves high performance across diverse tasks—like object detection or pixel-perfect reconstruction—without requiring any specialized training or adding decoding overhead.

What Problem Does This Paper Address?

Traditional image compression is usually optimized for human eyes, focusing on either pixel-wise fidelity (PSNR) or perceptual "pleasingness." However, in the age of AI, many images are compressed specifically for machine consumption—think surveillance feeds or autonomous vehicle data.

Existing "zero-shot" codecs (which use powerful, pretrained diffusion models to avoid expensive training) are often rigid; they are great at making images look "nice" but struggle to adapt when a specific task requires high pixel accuracy or the preservation of semantic features for an object detector. Previously, solving this meant training entirely new, task-specific codecs. Flex-DDCM addresses this by providing a way to "steer" a frozen diffusion model toward different goals on the fly.

Methods

Flex-DDCM builds upon Turbo-DDCM, a codebook-based diffusion compression method. In standard diffusion compression, the encoder selects noise vectors from a codebook that, when denoised, reconstruct the original image.

The "secret sauce" of Flex-DDCM is Task-guided Noise Selection. As shown in the overview below, the encoder doesn't just pick noise that matches the image; it calculates the gradient of a specific "task objective" (like a detection loss or MSE) and uses that gradient to bias its noise selection.

Fig. 1. Overview of Flex-DDCM for task-oriented image compression. Task- guided noise selection during encoding enables adaptation to different task requirements, such as pixel fidelity and object detection performance. This adaptation introduces no additional decoding complexity.

Key technical components include:

  • Trajectory Guidance: The encoder combines the standard reconstruction residual with local descent directions from differentiable task objectives. This "guided residual" determines which codebook atoms are selected.
  • Guidance Scheduling: Not all timesteps in the diffusion process are equal. The authors found that pixel fidelity benefits from guidance at later stages (fine details), while machine vision tasks perform better when guided by a "square-root SNR" profile.
  • Zero Decoding Overhead: Because the guidance happens entirely during the selection process at the encoder, the decoder receives standard noise indices. It simply runs the frozen diffusion model, unaware of the complex optimization that happened upstream.
Fig. 2. Task-guided noise selection in Flex-DDCM. Normalized task gradients are weighted and combined with the latent residual to select codebook atoms, whose indices and signs convey task guidance.

Datasets

The authors evaluated Flex-DDCM across three distinct domains:

  1. Kodak: The gold standard for evaluating signal fidelity (PSNR) and perceptual quality (LPIPS).
  2. COCO 2017: Used to test downstream machine vision tasks, specifically object detection (Faster R-CNN) and instance segmentation (Mask R-CNN).
  3. Flickr30K: Used for multimodal retrieval tasks (image-to-text and text-to-image) using the BEiT-3 model.

Key Insights and Ideas

  • Steering vs. Training: The most significant "aha moment" is that you don't need to retrain a generative model to change its fundamental behavior. By simply changing how you navigate its latent space (the "trajectory"), you can pivot from a perceptual codec to a machine-vision-optimized one.
  • The Importance of Timing: The paper highlights that different tasks live at different "frequencies" of the diffusion process. Semantic information for machine vision is often captured earlier in the denoising trajectory, whereas pixel-perfect reconstruction requires guidance at the very end.
  • Implicit Communication: The task preference is "baked into" the choice of noise. This allows for a "dumb" decoder that remains fast and efficient, even when the encoder is doing "smart" task-specific optimization.

Key Results

Flex-DDCM demonstrates a massive "adjustable range" compared to fixed codecs.

  • Performance: It consistently outperforms the base Turbo-DDCM in PSNR and achieves competitive results against state-of-the-art trained task-oriented codecs like Diff-ICMH.
  • Speed: While encoding is slower due to gradient calculations (~1.6s to 2.5s), the decoding time remains roughly 1 second for a 512x512 image. This is significantly faster than competitors like Diff-ICMH, which can take over 6 seconds to decode.
  • Flexibility: As seen in Figure 3, the "colored areas" represent the wide range of trade-offs the model can achieve simply by tuning the guidance strength ($\gamma$), allowing a single bitstream to serve multiple masters.
Fig. 3. Performance comparison on Kodak, COCO 2017, and Flickr30K. Colored areas represent the adjustable range of Flex-DDCM.

Limitations and Future Work

  • Encoding Complexity: The primary drawback is the increased computational load at the encoder. Calculating gradients through a diffusion UNet for every noise selection step is expensive, making it less suitable for real-time mobile encoding.
  • Dependence on Pretrained Priors: The method's success is tied to the quality of the underlying diffusion model (Stable Diffusion 2.1 in this case). If the model has biases or lacks certain textures, the guidance can only do so much to correct it.
  • Future Directions: The authors suggest exploring more efficient ways to compute guidance or extending the framework to video compression, where temporal consistency would add another layer of complexity to the trajectory guidance.

Summary updated

Abstract

Recent zero-shot image compression methods exploit the powerful priors of pretrained diffusion models to produce perceptually pleasing reconstructions without compression-specific training. However, image compression serves not only human viewing but also machine analysis, while existing zero-shot codecs have limited flexibility in meeting the diverse reconstruction requirements of these applications. In this paper, we propose Flex-DDCM, a framework for zero-shot task-oriented image compression via diffusion trajectory guidance. Building on Turbo-DDCM, we adapt the coded diffusion trajectory through task-guided noise selection, embedding the effects of task guidance into the compressed representation. Guidance terms can be applied individually or combined, with timestep-dependent strengths controlling their influence throughout denoising. This design enables task adaptation without introducing additional decoding complexity. Experiments demonstrate flexible trade-offs among pixel fidelity, perceptual quality, and downstream task performance, with competitive performance against trained task-oriented codecs and faster decoding than Diff-ICMH.
BibTeX
@misc{gao2026flex,
  author = {Yixin Gao and Runsen Feng and Xin Li and Zhibo Chen},
  title = {{Flex-DDCM: Zero-Shot Task-Oriented Image Compression via Diffusion Trajectory Guidance}},
  year = {2026},
  eprint = {2611.11427},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11427},
}
Figure