A Taxonomy of Miscompressions: Preparing Image Forensics for Neural Compression

Nora Hofer, Rainer Böhme

Abstract

As neural image compression (NIC) begins to outperform traditional codecs in perceptual quality, it introduces a novel forensic risk: the "miscompression." Unlike traditional compression artifacts that signal data loss through visible degradation, NIC can generate semantically incorrect reconstructions that appear visually flawless. This paper establishes a formal taxonomy of these failures, categorizing them by signal-processing characteristics and semantic impact to provide a framework for future forensic analysis and model hardening.

Introduction: The Authenticity Trap

In digital forensics, the reliability of an image is often gauged by its visual integrity. Traditional compression standards, such as JPEG, are "honest" about their failures; as bitrates drop, the image becomes progressively blurrier or blockier. These artifacts serve as a clear signal to human observers and automated systems that the data is low-quality and potentially unreliable.

Modern Neural Image Compression (NIC) breaks this paradigm. By leveraging generative architectures like Generative Adversarial Networks (GANs) and Conditional Diffusion Models (CDMs), NIC can maintain high perceptual quality even at extremely low bitrates. However, this visual clarity is often achieved through synthesis rather than reconstruction. The model "hallucinates" details to fill gaps in the compressed bitstream. When these hallucinations alter the meaning of the image—changing a digit on a license plate or removing a person from a scene—they create a "miscompression": a reconstruction that is visually convincing but semantically false.

Methodology

To understand the nature of these errors, researchers conducted an exploratory study involving the manual inspection of reconstructions from two state-of-the-art NIC models:

  • HiFiC: A GAN-based high-fidelity compression scheme.
  • CDC: A scheme based on conditional diffusion models.

The study utilized 552 images sourced from three standard benchmark datasets: CLIC2020, DIV2K, and Kodak. By comparing the original lossless images with their neurally compressed counterparts, the authors identified recurring patterns of semantic failure, which formed the basis for their provisional taxonomy.

A Taxonomy of Miscompressions

The paper distinguishes between a standard "artifact" (a visual distortion) and a "miscompression" (a semantic error). The proposed taxonomy categorizes these errors based on three primary signal-processing dimensions, supplemented by a high-impact modifier.

1. Amplitude

These miscompressions involve changes in brightness, color, or the intensity of high-frequency details. An example includes a car's brake lights appearing to be "off" in a reconstruction because the model failed to accurately synthesize the specific intensity of the red glow.

2. Geometry

Geometric miscompressions involve local shifts, rotations, or scaling of objects. In these instances, the object is present and recognizable, but its spatial properties are subtly altered—such as the profile of a person's face being reshaped or a sign being slightly tilted.

3. Shape

Shape-based errors occur when the actual form of an object is transformed. This is particularly dangerous in technical contexts, such as a traffic light's directional arrow being reconstructed as a simple round dot, fundamentally changing the interpreted instruction of the signal.

The "Symbol" Modifier

The authors propose a binary flag for any miscompression affecting symbols (letters, numbers, religious icons, or flags). Symbols are uniquely sensitive because minor pixel-level changes lead to massive shifts in human interpretation. A single pixel change can turn a '4' into a '6', leading to catastrophic failures in forensic identification.

Empirical Evidence of Semantic Failure

The study documented several startling qualitative failures that highlight the risks of NIC in real-world applications:

  • Numerical Alteration: Using the HiFiC model, the number 2264 on a sign was reconstructed as 2254. This error was so visually seamless that Google’s Cloud AI OCR read the hallucinated number as fact.
  • Object Erasure: In a reconstruction of a park scene, the NIC model completely omitted people who were present in the original image, effectively "cleaning" the scene of vital forensic evidence.
  • Symbolic Distortion: A Christian cross on a church tower was removed entirely, and a French flag was distorted into unrecognizable graffiti, demonstrating the model's struggle with culturally significant iconography.
  • Public Safety Risks: The transformation of directional traffic arrows into standard circular lights suggests that NIC could introduce liabilities in the analysis of traffic accidents or autonomous vehicle training data.

Conclusion and Future Directions

This taxonomy represents a first step toward "forensic hardening" for neural compression. The authors acknowledge that the definition of "semantic meaning" remains subjective and tied to cultural context. Furthermore, the manual nature of this study limits its scale.

Future work must focus on developing automated "safety nets"—detectors capable of flagging potential miscompressions during the encoding process. By identifying sensitive regions (like faces, text, or symbols), encoders could dynamically allocate more bits to those areas, ensuring that the reconstruction is not only visually pleasing but semantically accurate. As NIC moves toward wider adoption, establishing these forensic safeguards is essential to maintaining the trustworthiness of digital imagery.

Summary updated

BibTeX
@inproceedings{hofer2024taxonomy,
  author = {Nora Hofer and Rainer Böhme},
  title = {{A Taxonomy of Miscompressions: Preparing Image Forensics for Neural Compression}},
  booktitle = {WIFS},
  year = {2024},
  doi = {10.1109/wifs61860.2024.10810704},
}
Figure