Abstract
This paper addresses the challenge of efficiently compressing screen content images (SCIs) – computer generated images with unique attributes such as large uniform regions, sharp edges, and limited color palettes, which pose difficulties for conventional compression algorithms. We propose a Superpixel-based Content Aggregation Block (SCAB) to aggregate local pixels into one super-pixel and aggregate non-local information via super-pixel transformer. Such aggregation enables the dynamic assimilation of non-local information while maintaining manageable complexity. Furthermore, we enhance our channel-wise context entropy model with a Dynamic Feature Fusion (DFF) mechanism. This mechanism integrates decoded slices and side information dynamically based on their global correlation, allowing the network to dynamically learn the optimal weights for global information usage. Extensive experiments on three SCI datasets (SCID, CCT, and SIQAD) show our method’s superior RD performance and inference time, making it the first network comparable with the advanced VVC-SCC standard.
Introduction: The Challenge of Screen Content
Most image compression algorithms—both traditional (like JPEG) and modern AI-based ones—are optimized for "natural" images, which typically feature smooth gradients and noisy textures. However, Screen Content Images (SCIs), such as computer desktops, gaming streams, and online presentations, are fundamentally different. They contain large uniform color blocks, perfectly sharp edges (text), and repetitive patterns.
Standard AI models often struggle with SCIs because they rely on "local" processing, which fails to exploit global redundancies—for instance, a UI element on one side of the screen is often identical to one on the other. While global self-attention could solve this, it is computationally prohibitive for high-resolution images. This paper introduces a framework to bridge this gap efficiently.
Methodology
The proposed architecture follows a Variational Autoencoder (VAE) structure, enhanced by two primary innovations:
Superpixel-based Content Aggregation Block (SCAB)
Instead of performing expensive calculations on every single pixel, the SCAB clusters similar neighboring pixels into "superpixels." It then performs a Transformer-based attention operation in this reduced superpixel space. This allows the network to capture long-range dependencies (like repeating icons or text) across the entire image with manageable computational costs before mapping the information back to the original pixel grid.
Dynamic Feature Fusion (DFF)
In compression, the entropy model predicts pixel value likelihood to determine storage efficiency. We enhanced the standard channel-wise entropy model with a gating mechanism that dynamically weights decoded information and side-data based on their global correlation. This allows the model to learn the most efficient way to utilize global context for prediction.
Experimental Results
The model was trained on the SCI2K dataset and evaluated using three recognized benchmarks: SCID, CCT, and SIQAD.
Performance and Fidelity
The proposed method outperformed six leading AI-based compression models, achieving an average BD-rate saving of 34.66% against the VVC anchor. Notably, it is the first neural network to demonstrate performance comparable to VVC-SCC, the specialized industrial standard for screen content.
Qualitatively, the model preserves sharp text and fine lines much better than previous methods, which often introduced blurring or "ringing" artifacts around letters (as seen in the detail of the 'O' and 'V' characters in Figure 2). Despite the use of Transformers, the superpixel approach maintains inference times competitive with other high-performance neural codecs.
Conclusion
This work marks a significant milestone in SCI compression by achieving parity with specialized industrial standards. While the superpixel clustering adds some iterative overhead during encoding, the gains in visual fidelity and compression efficiency are substantial. Future research will explore extending these dynamic fusion and superpixel blocks to temporal video compression and hybrid content containing both natural and synthetic elements.
Summary updated