Unsupervised Categorization (Filtering) of Google Images Based on Visual Consistency

Pooyan Fazli, Ara BedrosianAAAI 2008

Abstract

This paper presents an unsupervised method to filter and re-rank noisy image search results by identifying "visual consistency." By rewarding frequently occurring visual features in search results and penalizing features found in "clutter" images, the system creates a clean training set for mobile robots without requiring manual data labeling.

Introduction

In the late 2000s, image search engines relied primarily on text cues—filenames, ALT tags, and surrounding HTML—rather than visual content. This often resulted in "noisy" datasets where a search for a specific term might return logos, clip art, or unrelated photos. For the Semantic Robot Vision Challenge (SRVC), this noise is a significant hurdle. A robot entering a new environment needs to download images of objects and train a classifier autonomously. If the training data is heavily contaminated, the robot will fail to recognize real-world objects. The authors address this by automatically "cleaning" search results at speeds suitable for real-time mobile robot deployment.

Raw image search results containing significant noise, including unrelated backgrounds and buildings.

Methodology

The proposed pipeline is based on the heuristic of Visual Consistency: relevant images of a specific object share common visual features, whereas "noise" images are visually unique or inconsistent with the rest of the set. The system first downloads "Positive Data" from Google and "Negative Data" from random clutter databases.

Using the DoG-SIFT (Difference of Gaussian - Scale-Invariant Feature Transform) detector, the system extracts keypoints robust to scale and rotation. These features are organized in a flat histogram via a KD-Tree for efficient similarity searching. Each feature is scored based on its popularity across the positive set and penalized if it appears in the negative clutter set. Finally, images are re-ranked based on the cumulative scores of their features; consistent images move to the top, while outliers are discarded.

Key Results and Insights

The core insight is that while noise in web searches is high, it is also high-entropy (random), whereas the signal (the actual object) is low-entropy (repetitive). By counting repeating visual patterns, the system distinguishes the subject from background noise. This "flat histogram" approach prioritizes efficiency over complexity, making it suitable for mobile robots with limited onboard computing.

Refined image set where consistent features (red circles) are identified, allowing for the re-ranking of relevant images to the top.

Qualitatively, the system effectively moves "clean" instances of objects, such as human faces, to the top of the results. The system proved robust to variations in viewpoint and pose, handling raw Google Image results with acceptable accuracy for autonomous robot learning.

Limitations and Future Work

The current "bag of features" approach ignores spatial context—the geometric arrangement of features (e.g., eyes relative to a nose). Future work aims to incorporate spatial configuration and more sophisticated clustering algorithms. Additionally, integrating linguistic data from source webpages could further refine the filtering process by combining visual consistency with linguistic relevance.

Summary updated

BibTeX
@inproceedings{fazli2008unsupervised,
  author = {Pooyan Fazli and Ara Bedrosian},
  title = {{Unsupervised Categorization (Filtering) of Google Images Based on Visual Consistency}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2008},
}
Figure