Abstract
Standard image compression standards, such as Versatile Video Coding (VVC), are primarily optimized for human perceptual quality. However, as machine vision tasks like autonomous driving and surveillance become more prevalent, traditional metrics like Mean Square Error (MSE) often fail to preserve the semantic features necessary for AI models. This paper introduces a machine-vision-optimized rate-distortion model that prioritizes bit allocation for regions critical to machine analysis. By incorporating a Region of Interest for Machine (ROIM) map and a Multi-Scale Feature Distortion (MSFD) metric into the VVC framework, the proposed method achieves significant bitrate reductions—up to 28.17%—while maintaining high performance on tasks such as object detection and semantic segmentation.
Introduction
Most existing image codecs are designed to minimize pixel-level distortion or maximize human-perceived quality. While effective for human consumption, these methods can be detrimental to Convolutional Neural Networks (CNNs). At low bitrates, standard compression artifacts can lead to a significant drop in accuracy; for instance, VGG-19 accuracy on ImageNet can fall significantly when bitrates are low (e.g., ~68% Top-5 accuracy at 0.1 bpp). Machines prioritize semantic features over visual smoothness. This research addresses the gap by optimizing the Versatile Video Coding (VVC) standard for machine vision tasks.
Proposed Methodology
The proposed framework integrates three core innovations into the VTM encoder to bridge the gap between compression and machine analysis.
1. Region of Interest for Machine (ROIM)
The system utilizes a pre-trained Region Proposal Network (RPN) to generate importance maps. By calculating the density of bounding boxes for each Coding Tree Unit (CTU), the encoder identifies regions most critical for downstream machine tasks.
2. Visual Analysis-Friendly Bit Allocation
Using ROIM data, the encoder intelligently distributes the bit budget. Critical regions, such as vehicles or pedestrians, receive higher bitrates, while background elements like the sky are compressed more aggressively. A "connectivity" constraint is also applied to maintain consistent Quantization Parameters (QP) across object boundaries, preventing artifacts that might confuse AI models.
3. Multi-Scale Feature Distortion (MSFD)
Traditional Rate-Distortion Optimization (RDO) relies on MSE. This paper introduces MSFD, a deep-feature-based metric. Since standard VVC coding units (as small as 4x4) lack sufficient spatial context for CNNs, the MSFD approach uses a multi-scale window to incorporate surrounding reconstructed pixels. This expands the receptive field, allowing the model to calculate distortion based on the preservation of high-level features extracted via a VGG-11 network.
Experimental Results
The model was evaluated across three tasks: image classification (ImageNet), object detection (VOC-2007), and semantic segmentation (COCO-2014).
The proposed scheme significantly outperforms the VVC test model (VTM 10.1) in terms of BD-Rate savings for equivalent accuracy levels:
- Semantic Segmentation: 28.17% reduction.
- Object Detection: 26.25% reduction.
- Image Classification: ~19% reduction.
Qualitative analysis shows that the model successfully preserves identifying features that standard VVC might blur, preventing misclassifications (e.g., mistaking an iPod for a television) at low bitrates.
Conclusion and Future Work
This research demonstrates that optimizing image compression for semantic feature preservation rather than pixel fidelity can yield substantial efficiency gains for machine vision. While currently limited to VVC intra-frame coding, future work will explore extending this model to inter-frame video coding. Additionally, reducing the computational overhead of the CNN-based RDO loop remains a key area for optimization.
Summary updated