Divergent Search for Few-Shot Image Classification

Jeremy Tan, Bernhard Kainz

Abstract

When data is unlabelled and the target task is not known a priori, divergent search offers a strategy for learning a wide range of skills. Having such a repertoire allows a system to adapt to new, unforeseen tasks. Unlabelled image data is plentiful, but it is not always known which features will be required for downstream tasks. We propose a method for divergent search in the few-shot image classification setting and evaluate with Omniglot and Mini-ImageNet. This high-dimensional behavior space includes all possible ways of partitioning the data. To manage divergent search in this space, we rely on a meta-learning framework to integrate useful features from diverse tasks into a single model. The final layer of this model is used as an index into the 'archive' of all past behaviors. We search for regions in the behavior space that the current archive cannot reach. As expected, divergent search is outperformed by models with a strong bias toward the evaluation tasks. But it is able to match and sometimes exceed the performance of models that have a weak bias toward the target task or none at all. This demonstrates that divergent search is a viable approach, even in high-dimensional behavior spaces.

Introduction

Most unsupervised learning methods for images rely on "proxy tasks"—such as predicting rotations or clustering—to create artificial labels. These methods typically exhibit a strong "convergent" bias, attempting to force the model to learn features that align with human-defined classes. However, in real-world scenarios, the target task is often unknown a priori. An image of a dog in a park could be classified by breed, pose, background vegetation, or weather conditions. Convergent algorithms usually settle on one of these perspectives and ignore the others. This paper addresses the challenge of Divergent Discriminative Feature Accumulation (DDFA): systematically searching for and archiving as many distinct ways of partitioning data as possible.

Methodology

The proposed method builds upon Reptile, a first-order meta-learning algorithm, replacing supervised tasks with a search for novelty. The framework operates through a dual-loop system:

The system maintains an Archive model representing its current knowledge. Its final layer serves as an index to past behaviors. A Teacher model is then tasked with finding a "New Behavior"—a way of partitioning a batch of images that the Archive cannot currently replicate. This is achieved by maximizing the Jensen-Shannon (JS) divergence between the Teacher's and the Archive's outputs. Once a novel behavior is identified, a Learner model is trained to master it.

The Outer Loop (The Integration)

Gradients from multiple Learners, each representing a different novel behavior, are aggregated to update the global model parameters. This update uses a weighted average based on Novelty (difficulty for the archive to match) and Quality/Robustness (stability under input noise). This process moves the global model toward a state that can easily adapt to any discovered behavior.

The Divergent Search framework: The Inner Loop discovers novel behaviors outside the current archive, while the Outer Loop integrates these behaviors into the global model.

Evaluation

The method was evaluated on two standard few-shot learning benchmarks: Omniglot, containing 1,623 handwritten characters, and Mini-ImageNet, featuring 100 classes of complex natural scenes.

Sample images from the Mini-ImageNet dataset illustrating the complexity of the natural scenes used for evaluation.

Key Results

In a 5-way, 5-shot classification setting, the proposed method demonstrated significant improvements over baselines:

  • Mini-ImageNet: The method achieved 43.41% accuracy, outperforming Random Initialization (36.52%) and Fixed Random Labels (32.49%). It performed comparably to UMTRA (an unsupervised meta-learning baseline) under low-augmentation settings.
  • Omniglot: The method reached 77.85% accuracy, a substantial increase over Random Initialization (64.29%).

Qualitative findings show that while supervised models cluster by object category, the divergent search model clusters by visual properties like the "busyness" of an image or the orientation of an object.

Discussion

A key insight of this work is the use of Novelty as an Objective. Unlike clustering, which seeks a "correct" partition, divergent search treats any novel partition as potentially valuable. This tension between the Archive and the Teacher creates a dynamic frontier for learning generic features like spatial frequency, pose, or style, rather than just object categories. This approach avoids the "random label trap," where training on noise forces memorization; instead, tasks are grounded in the model's current capabilities.

While divergent search is currently outperformed by models with strong task-specific biases, it proves effective in high-dimensional behavior spaces. Future work will focus on refining quality metrics to better distinguish useful novelty from noise and exploring larger model architectures to store more extensive behavior repertoires.

Summary updated

Abstract

When data is unlabelled and the target task is not known a priori, divergent search offers a strategy for learning a wide range of skills. Having such a repertoire allows a system to adapt to new, unforeseen tasks. Unlabelled image data is plentiful, but it is not always known which features will be required for downstream tasks. We propose a method for divergent search in the few-shot image classification setting and evaluate with Omniglot and Mini-ImageNet. This high-dimensional behavior space includes all possible ways of partitioning the data. To manage divergent search in this space, we rely on a meta-learning framework to integrate useful features from diverse tasks into a single model. The final layer of this model is used as an index into the `archive' of all past behaviors. We search for regions in the behavior space that the current archive cannot reach. As expected, divergent search is outperformed by models with a strong bias toward the evaluation tasks. But it is able to match and sometimes exceed the performance of models that have a weak bias toward the target task or none at all. This demonstrates that divergent search is a viable approach, even in high-dimensional behavior spaces.
BibTeX
@inproceedings{tan2020divergent,
  author = {Jeremy Tan and Bernhard Kainz},
  title = {{Divergent Search for Few-Shot Image Classification}},
  booktitle = {GECCO Companion},
  year = {2020},
  doi = {10.1145/3377929.3389973},
}
Figure