Introduction
Traditional image compression standards like JPEG or VVC prioritize pixel-wise accuracy, often measured by Mean Squared Error (MSE). While effective at moderate bitrates, these methods suffer from severe artifacts—blurring and blockiness—when pushed to extreme compression levels. This paper introduces Hierarchical Semantic Compression (HSC), a framework designed to maintain visual and semantic consistency at ultra-low bitrates by operating within the intrinsic semantic spaces of generative models like StyleGAN.
The HSC Framework
The HSC framework moves away from pixel-matching toward preserving the "meaning" or "concept" of an image. It leverages the pre-learned, disentangled latent space of StyleGAN through three primary components:
- General Inversion Encoder (GIE): This module inverts the input image into the StyleGAN space, extracting high-level core semantics ($s$) and middle-level features ($f$).
- Semantic Compression Network (SCN): This network compresses the core latent codes. Because these codes are highly disentangled (e.g., separating pose from hair color), they represent the image's essence with minimal data.
- Feature Compression Network (FCN): To capture details beyond the core semantics, the FCN compresses mid-level features using a Semantic Context Model. This model uses the high-level "concept" as a prior for channel-wise autoregressive entropy modeling, ensuring structural details are compressed efficiently.
Key Results and Perceptual Quality
HSC significantly outperforms traditional codecs and existing neural compressors in perceptual metrics at ultra-low bitrates. On the CelebA-HQ dataset, HSC achieved a Fréchet Inception Distance (FID) of 12.93, surpassing state-of-the-art methods like HiFiC.
A unique strength of HSC is its "semantic floor." Even at bitrates as low as 0.003 bpp, where traditional methods fail completely, HSC maintains a recognizable and semantically consistent reconstruction by relying on the GAN's learned priors.
Beyond human perception, HSC also excels in machine vision tasks. In face segmentation tests, it maintained high accuracy (FWIoU > 0.8) at bitrates where standard codecs' performance dropped below 0.4.
Editable Bitstreams and Future Work
Because HSC compresses images directly into a generative latent space, the resulting bitstream is "live." Users can perform semantic editing—such as adding a smile or changing hair color—by modifying the latent codes within the compressed data without full decompression.
Currently, the framework is domain-specific, requiring a pre-trained StyleGAN for each category (e.g., faces, cars). Future research aims to scale this approach to more general-purpose generative models, such as Diffusion Models, to handle complex, multi-object scenes.
Summary updated