TL;DR
Deep learning has shown promise in image compression, but its perceptual quality hasn't been thoroughly evaluated. This paper bridges that gap by conducting a rigorous subjective quality study on learned compression algorithms, finding that MS-SSIM optimization yields competitive results and offers valuable insights for future developments.
What Problem Does This Paper Address?
Traditional image compression methods like JPEG and HEVC rely on hand-crafted transforms and quantization. While effective, the advent of high-resolution images and novel formats necessitates better compression. Deep learning has emerged as a powerful tool, with autoencoder-based approaches achieving impressive compression ratios. However, most deep learning compression research focuses on objective quality metrics (like PSNR or MS-SSIM) and rarely includes comprehensive subjective evaluations. This leaves a critical gap: how do these learned compression artifacts actually impact human perception, especially given that they differ from traditional artifacts? This paper aims to address this by providing a much-needed subjective quality assessment of learned image compression techniques.
Methods
The authors designed a general learned image compression framework based on a convolutional autoencoder (CAE). This CAE architecture includes an analysis transform, quantization, and a synthesis transform, all trained end-to-end. They explored two primary objective loss functions for training: Mean Squared Error (MSE) and Multi-Scale Structural Similarity (MS-SSIM). For handling high-resolution images, they employed a tiling strategy with a 32-pixel overlap to mitigate redundancy and artifacts.
For the subjective quality evaluation, they adopted the Absolute Category Rating with Hidden Reference (ACR-HR) methodology. This involved presenting participants with images compressed by various codecs at different bitrates and asking them to rate the visual quality on a five-level scale. The experiment was conducted in a controlled environment with calibrated displays and a group of 16 participants.
Datasets
The paper utilized the Kodak dataset for initial performance comparisons (Fig. 3). For the subjective quality evaluation, they selected seven uncompressed 8-bit RGB test images with high resolutions (HD to UHD), as specified in the JPEG XL Call for Proposals. These images are referred to as APPLE, ARRI, BIKE, CAFE, FemaleStripedHorseFly, P06, and Woman.
Key Insights and Ideas
- MS-SSIM is Key for Perceptual Quality: Optimizing learned compression models with MS-SSIM, a metric that better aligns with human visual perception than MSE, leads to significantly better subjective quality, especially at lower bitrates.
- Learned Artifacts are Different: The paper highlights that deep learning compression introduces novel artifacts that are distinct from the blocking and ringing artifacts of traditional codecs. While these new artifacts might preserve shapes better, they can sometimes make textures (like rocks) appear unnatural.
- Tiling Strategy Matters: Using an overlapping tiling strategy for high-resolution images proved to be superior to non-overlapping tiling for learned compression.
- Objective vs. Subjective Mismatch: The study demonstrates a poor correlation between objective metrics (PSNR, MS-SSIM) and subjective quality ratings for some learned compression approaches, underscoring the need for better perceptual similarity metrics in training.
Key Results
- The learned compression algorithm optimized by MS-SSIM achieved competitive results, often approaching the performance of state-of-the-art codecs like HEVC/H.265-intra in subjective evaluations.
- Specifically, the CAE-MS-SSIM-ov model was found to be the second-best performer overall, statistically comparable to HEVC/H.265-intra at certain bitrates.
- For face-related content (e.g., "Woman"), the MS-SSIM optimized learned compression even outperformed HEVC/H.265-intra in subjective quality.
- The study found that MS-SSIM optimization was more effective than MSE optimization for achieving higher subjective quality at low bitrates.
Limitations and Future Work
- Limited Number of Participants: While 16 participants were involved, a larger and more diverse group could provide even more robust results.
- Specific Learned Architectures: The study focused on a specific CAE architecture. Future work could explore other deep learning models.
- Need for Better Perceptual Metrics: The authors explicitly state the need for improved perceptual similarity metrics to better guide the training of learned image compression models, as current objective metrics don't always align with subjective perception.
- Further Exploration of Artifacts: More in-depth analysis of the specific types of artifacts introduced by learned compression and their perceptual impact would be beneficial.
Summary updated