ImagiFilter: A resource to enable the semi-automatic mining of images at scale

Houda Alberts, Iacer CalixtoarXiv 2020 · arXiv

Abstract

In the era of "Big Data," collecting millions of images from the web is straightforward, but ensuring dataset quality remains a significant challenge. ImagiFilter is an open-source dataset and suite of models designed to automatically clean "noisy" image collections by distinguishing between natural photographs and undesirable content such as maps, icons, and sketches. This paper introduces the ImagiFilter dataset of 6,000 annotated images and evaluates several Convolutional Neural Network (CNN) architectures. Our results show that even lightweight models can achieve over 96% accuracy in binary filtering, while deeper architectures are required for fine-grained classification.

Introduction

Building robust computer vision models for natural object recognition requires high-quality training data. However, large-scale web-scraped datasets are often cluttered with non-photographic "noise"—weather maps, hand-drawn sketches, or UI icons—that can degrade model performance. Traditionally, researchers have relied on expensive manual curation or crowdsourcing to filter this noise. ImagiFilter addresses this need by providing an automated tool for "coarse" filtering (separating photos from non-photos) and "fine-grained" classification of non-photographic content.

Examples of photographic content versus non-photographic noise like icons and maps.

Methods

The authors frame image filtering as a classification task using three CNN architectures:

  1. ResNet-152 and VGG-19: Deep, heavyweight models pre-trained on ImageNet and fine-tuned for filtering.
  2. LeNet-5: A lightweight architecture trained from scratch to evaluate efficiency for high-speed pipelines.

The models were evaluated on two levels: Coarse prediction (binary classification: photo vs. non-photo) and Fine-grained prediction (identifying specific non-photo categories).

The ImagiFilter Dataset

The ImagiFilter dataset consists of 6,000 manually annotated images sourced via the BabelNet API. The dataset is balanced, containing 3,000 "positive" (photographic) and 3,000 "negative" (non-photographic) examples. The negative class is further categorized into specific types to support more nuanced filtering.

Fine-grained categories in the ImagiFilter dataset, including graphs, flags, and sketches.

The fine-grained labels include:

  • Maps: 1,042 images
  • Icons: 810 images
  • Graphs/Rendered images: 425 images
  • Flags: 372 images
  • Hand drawings/Sketches: 157 images
  • Other: 206 images

Key Insights and Results

Model Performance

All tested models performed exceptionally well at binary (coarse) classification. ResNet-152 achieved the highest accuracy at 98.8%, followed by VGG-19 (98.4%) and LeNet-5 (96.3%). For fine-grained tasks, accuracies ranged from 87% to 98%, with ResNet-152 proving most consistent due to its ability to leverage pre-trained features for underrepresented classes like "sketches."

Efficiency vs. Complexity

A key finding is the effectiveness of the lightweight LeNet-5. Despite its small memory footprint and age, its 96.3% accuracy makes it ideal for mobile deployment or high-throughput data processing where computational resources are constrained.

Challenges in Edge Cases

Error analysis revealed that models struggle with very dark or black-and-white images. Artistic photographs with low lighting are sometimes misclassified as non-photographic noise, indicating a need for better handling of lighting extremes.

Conclusion and Future Work

ImagiFilter provides a scalable solution for the semi-automatic curation of image datasets. By automating the removal of non-photographic noise, it significantly reduces the manual effort required for large-scale data mining. Future work will focus on expanding the dataset to address class imbalances—particularly for sketches—and improving robustness against edge cases like logos and low-light photography.

The dataset and code are available at https://github.com/houda96/imagi-filter.

Summary updated

Abstract

Datasets (semi-)automatically collected from the web can easily scale to millions of entries, but a dataset's usefulness is directly related to how clean and high-quality its examples are. In this paper, we describe and publicly release an image dataset along with pretrained models designed to (semi-)automatically filter out undesirable images from very large image collections, possibly obtained from the web. Our dataset focusses on photographic and/or natural images, a very common use-case in computer vision research. We provide annotations for coarse prediction, i.e. photographic vs. non-photographic, and smaller fine-grained prediction tasks where we further break down the non-photographic class into five classes: maps, drawings, graphs, icons, and sketches. Results on held out validation data show that a model architecture with reduced memory footprint achieves over 96% accuracy on coarse-prediction. Our best model achieves 88% accuracy on the hardest fine-grained classification task available. Dataset and pretrained models are available at: https://github.com/houda96/imagi-filter.
BibTeX
@misc{alberts2020imagifilter,
  author = {Houda Alberts and Iacer Calixto},
  title = {{ImagiFilter: A resource to enable the semi-automatic mining of images at scale}},
  year = {2020},
  eprint = {2008.09152},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2008.09152},
}
Figure