An Agentic Framework for Quality-Aware and Scenario-Adaptive Image Compression

Zhenghao Chen, Lebin Zhou, Zicong Chen, Naifu Xue, Jiahao Li, Bin Li, Dong Xu, Yan Lu

TL;DR

Practical Image Coding Agents (PICA) is a training-free, multi-agent framework that makes existing image codecs "smart" by adapting them to diverse user requirements and degraded inputs. By treating codecs as black boxes and using a "schedule-execute-reflect" loop, PICA consistently outperforms standard and specialized codecs across perceptual, machine-vision, and standard compression scenarios without requiring any architectural redesign or retraining.

What Problem Does This Paper Address?

Traditional image codecs (like JPEG or H.266/VVC) and even modern neural image codecs (NIC) suffer from two major rigidities. First, they are typically optimized for a single objective—usually pixel-level fidelity (PSNR)—making them suboptimal for specialized tasks like human perceptual realism or downstream machine-vision tasks (e.g., object detection). Second, they assume "clean" input images. In reality, sensor noise or motion blur can confuse a codec, causing it to waste precious bits encoding artifacts rather than the actual signal.

Adapting a codec to these variables usually requires massive datasets and expensive retraining. The authors ask: Can we adapt existing, widely-used codecs to diverse scenarios and degraded inputs autonomously, without touching their internal architecture?

Methods

The authors introduce PICA, a framework comprising four collaborative agents that manage the compression pipeline through a Scenario-Guided Quality–Rate (SGQR) evaluation strategy.

Fig. 1: Overview of the Practical Image Compression Agent (PICA) framework, consisting of four collaborative agents: the Perception agent, the Improvement agent, the Coding agent, and the After-coding improvement agent.
  1. Perception Agent: Uses Vision-Language Models (VLMs) to interpret user intent (e.g., "compress for object detection") and analyze the image for degradations and Regions of Interest (ROIs).
  2. Improvement Agent (Pre-coding): Performs Quality-Rate (QR)-aware restoration. It doesn't just "fix" the image; it evaluates whether denoising or deblurring will actually improve the final bitrate-to-quality trade-off.
  3. Coding Agent: This is the core of the framework. It performs a global search for the best quality settings and utilizes a novel Agentic ROI Coding (AROIC) mechanism. AROIC treats the codec as a black box, cropping ROIs, encoding them with higher quality, and reassembling the image. This brings ROI capabilities to codecs that don't natively support them.
  4. After-Coding Improvement Agent: A final pass to clean up residual compression artifacts or degradations using quality-only metrics.
Fig. 2: The Coding Agent workflow comprises two components: Global QC Search and Agentic ROI Coding (AROIC).

The system operates on a reflection loop: it proposes a coding plan, executes it using the target codec, evaluates the result against scenario-specific metrics (like LPIPS for humans or mAP for machines), and iterates until the goal is met.

Datasets

The framework was evaluated on three major benchmarks, with synthetic Gaussian noise and defocus blur added to test robustness:

  • Kodak: 24 natural images for standard and perceptual evaluation.
  • CLIC2020: 102 high-resolution images for varied visual content.
  • HEVC (Classes C and D): Used for machine-vision evaluation, specifically object detection performance using the SFU-HW-Objects-v3 annotations.

Key Insights and Ideas

  • The "Black Box" Advantage: By treating the codec as a black box, PICA is future-proof. It can be wrapped around H.266, a new neural codec, or even a legacy standard like JPEG without modification.
  • Restoration as a Bitrate Saver: A key "aha" moment is that pre-coding restoration (like denoising) isn't just for aesthetics. By removing noise before the codec sees the image, PICA prevents the codec from wasting bits on high-frequency "junk," significantly improving compression efficiency.
  • Agentic ROI Coding (AROIC): Most ROI methods are baked into the codec's math. AROIC proves you can achieve similar results through agentic scheduling—deciding which parts of the image to crop and how to independently encode them for the best aggregate result.
  • Scenario-Specific Metrics: PICA changes its "definition of success" based on the user. For machine vision, it optimizes for task accuracy; for humans, it optimizes for perceptual metrics like DISTS and LPIPS.

Key Results

PICA was tested using both the conventional H.266/VVC and the neural Cheng2020 codec as base engines.

  • Perceptual Compression: On the Kodak dataset, PICA achieved average bitrate savings of 34.5% to 62.6% over the baseline Cheng2020 codec at matched perceptual quality (LPIPS).
  • Machine Vision: For object detection on HEVC Class D, PICA showed relative gains in mAP (mean Average Precision) of up to 80.4% compared to standard VVC at the same bitrate.
  • Robustness: PICA outperformed specialized codecs like HiFiC (perceptual) and DenoiseCompression (noisy inputs) when those codecs were forced to operate outside their specific training niches.
  • VLM Agnostic: The framework proved robust across different VLM backbones, including Qwen2.5-VL and GPT-4o, yielding nearly identical performance gains.

Limitations and Future Work

The primary limitation is computational overhead. Because PICA uses an iterative "schedule-execute-reflect" loop involving multiple encodings and VLM inferences, it is currently slower than a single-pass codec.

Future work suggested by the authors includes:

  • Optimizing the efficiency of the agentic loops for real-time applications.
  • Extending the framework to video compression, where temporal consistency adds a new layer of complexity.
  • Exploring more sophisticated VLM-based feedback to further refine regional bit allocation.

Summary updated

Abstract

Despite decades of progress, image codecs remain difficult to adapt to diverse real-world compression scenarios for two main reasons. First, most image codecs are optimized for a specific objective and require substantial redevelopment when the compression goal changes. Second, they typically assume clean inputs, whereas sensor-captured images often contain degradations that alter rate-quality behavior. To address these challenges, we propose Practical Image Coding Agents (PICA), a multi-agent framework that adapts existing codecs to diverse user requirements and potentially degraded inputs. PICA consists of four collaborative agents: a Perception agent that interprets user intent and analyzes image content, an Improvement agent that jointly considers bitrate and reconstruction quality to guide pre-coding restoration, a Coding agent that refines quality-control settings from global to regional levels, and an After-coding improvement agent that addresses residual degradations and compression-induced artifacts through restoration guided solely by image quality. To enable codec-agnostic regional adaptation, we introduce Agentic ROI Coding (AROIC), which treats the codec as a black box and allows the agent to automatically schedule, execute, evaluate, and reflect ROI coding plans without architectural modifications or codec-specific retraining. We integrate these four agents with a Scenario-Guided Quality-Rate (SGQR) evaluation strategy to enable more effective scenario-adaptive compression across heterogeneous codecs. Comprehensive experiments on clean, noisy, and blurred images across standard, perceptual, and machine-vision compression scenarios demonstrate that PICA consistently enhances the compression performance of both the conventional H.266/VVC codec and the neural Cheng2020 codec, outperforming competing compression methods on the Kodak, CLIC2020, and HEVC datasets.
BibTeX
@misc{chen2026agentic,
  author = {Zhenghao Chen and Lebin Zhou and Zicong Chen and Naifu Xue and Jiahao Li and Bin Li and Dong Xu and Yan Lu},
  title = {{An Agentic Framework for Quality-Aware and Scenario-Adaptive Image Compression}},
  year = {2026},
  eprint = {2611.04960},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04960},
}
Figure