Low-Quality Image Detection by Hierarchical VAE

Tomoyasu Nanaumi, Kazuhiko Kawamoto, Hiroshi KeraarXiv 2024 · OpenReview

Abstract

To make an employee roster, photo album, or training dataset of generative models, one needs to collect high-quality images while dismissing low-quality ones. This study addresses a new task of unsupervised detection of low-quality images. We propose a method that not only detects low-quality images with various types of degradation but also provides visual clues of them based on an observation that partial reconstruction by hierarchical variational autoencoders fails for low-quality images. The experiments show that our method outperforms several unsupervised out-of-distribution detection methods and also gives visual clues for low-quality images that help humans recognize them even in thumbnail view.

Introduction: The Challenge of Data Cleaning

In the era of massive datasets, "data cleaning" has become a significant bottleneck. Whether building a facial recognition roster or training a generative AI like Stable Diffusion, the quality of the input data is paramount. However, low-quality images—those suffering from subtle Gaussian noise, JPEG artifacts, or slight motion blur—are often difficult to detect automatically.

The challenge is two-fold. First, there is an infinite "diversity of decay." Images can be corrupted in countless ways (blur, noise, lighting, etc.), making it nearly impossible to train a supervised classifier that catches every unseen type of corruption. Second, there is the "thumbnail problem": many corruptions are nearly invisible in gallery or thumbnail views, requiring time-consuming manual inspection where humans must zoom in on every photo to verify quality.

A low-quality image where subtle artifacts are only visible upon close inspection.

Methodology: Leveraging Hierarchical VAEs

The proposed method leverages Hierarchical Variational Autoencoders (VAEs), specifically the "Very Deep VAE" (VDVAE) architecture. Unlike standard VAEs that compress an image into a single latent representation, hierarchical VAEs operate across many layers. This structure allows the model to capture global features (like face shape) in higher layers and fine-grained details (like skin texture) in lower layers.

Partial Reconstruction and the "Breakdown" Observation

The core of the methodology is a process called Partial Reconstruction. The model encodes an image and then attempts to reconstruct it. During reconstruction, the actual encoded data is used for the "top" layers (global structure), but the "bottom" layers (fine details) are filled in using a random normal distribution (the prior).

For clean images, this partial reconstruction results in a plausible, slightly smoothed version of the original. However, for corrupted images, the process fails spectacularly. Because the model was trained only on clean data, it cannot reconcile the global structure of a noisy image with its learned internal logic, producing distorted, "alien-looking" visual clues.

A distorted "alien-looking" reconstruction serves as a clear visual clue of image corruption.

Scoring and Adaptive Selection

Detection is quantified using a score function based on the Kullback–Leibler (KL) divergence between the latent distributions of the original image and its partial reconstruction. A high divergence indicates a likely low-quality image. Furthermore, because different corruptions affect different frequencies, the researchers use a Fast Fourier Transform (FFT) to adaptively choose the optimal layer at which to split the reconstruction, enhancing detection for both high-frequency noise and low-frequency blurs.

Experimental Results

The method was evaluated using the FFHQ-256 (high-quality faces) and ImageNet-64 datasets. To test robustness, 19 types of synthetic corruption from the "ImageNet-C" benchmark were applied at "Severity Level 1"—the subtlest and most difficult level to detect.

On the FFHQ dataset, the proposed method achieved an AUROC of 91.3% and a False Positive Rate (FPR80) of 14.4%. This significantly outperformed established Out-of-Distribution (OOD) detection methods such as PANDA (79.5% AUROC) and CSI (78.8% AUROC). The method was particularly effective at catching Gaussian noise, various blurs, and JPEG compression artifacts, often achieving near-perfect AUROC scores in these categories.

Conclusion

This research demonstrates that low-quality images can be effectively detected without labeled "bad" examples by treating corruption as a violation of a learned structural hierarchy. The resulting "visual clues" are not just mathematical signals but practical tools; by presenting these distorted reconstructions alongside thumbnails, human annotators can immediately identify defects that would otherwise require manual zooming. While computational costs for deep VAEs remain a factor, the unsupervised nature and high performance of this approach offer a promising path for automated large-scale data curation.

Summary updated

Abstract

To make an employee roster, photo album, or training dataset of generative models, one needs to collect high-quality images while dismissing low-quality ones. This study addresses a new task of unsupervised detection of low-quality images. We propose a method that not only detects low-quality images with various types of degradation but also provides visual clues of them based on an observation that partial reconstruction by hierarchical variational autoencoders fails for low-quality images. The experiments show that our method outperforms several unsupervised out-of-distribution detection methods and also gives visual clues for low-quality images that help humans recognize them even in thumbnail view.
BibTeX
@misc{nanaumi2024low,
  author = {Tomoyasu Nanaumi and Kazuhiko Kawamoto and Hiroshi Kera},
  title = {{Low-Quality Image Detection by Hierarchical VAE}},
  year = {2024},
  eprint = {2408.10885},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.10885},
}
Figure