Meta-Learning for Batch Mode Active Learning

Sachin Ravi, Hugo LarochelleWorkshops @ ICLR 2018 · OpenReview

Overview

Instead of relying on hand-crafted heuristics to pick which data to label, this paper proposes a meta-learning approach that learns an active learning algorithm. Specifically designed for "batch mode" (picking multiple items at once) and few-shot scenarios, the proposed model outperforms standard methods even when the unlabeled data is cluttered with irrelevant "distractor" items.

Problem Statement

Active learning is the art of choosing the most informative unlabeled data points to send to a human for labeling. Most popular methods use simple heuristics—for example, "pick the image the model is most confused about" (Max-Entropy).

However, the authors identify three major gaps in current research:

  1. The Batch Problem: In real-world applications, practitioners do not label one image, retrain the whole model, and repeat. They label in batches. Simple heuristics often pick redundant items in a batch (e.g., five slightly different photos of the same dog), which wastes human effort.
  2. Computational Complexity: Existing algorithms designed specifically for batches are often mathematically heavy, scaling quadratically with the size of the unlabeled pool. This makes them too slow for large datasets.
  3. The Distractor Problem: Most research assumes every unlabeled item belongs to a class of interest. In reality, unlabeled data is messy and contains "distractors"—items that don't belong to any target class. Heuristics often mistakenly flag these as "informative" because the model is uncertain about them, wasting the labeling budget.

Methodology

The authors frame active learning as a meta-learning problem. They build their system on top of Prototypical Networks, a popular framework for few-shot learning where classes are represented by a "prototype" (the mean vector of its examples).

The core of their contribution is a learned selection policy trained via Reinforcement Learning (specifically the REINFORCE algorithm). To pick a batch of size $B$, the model uses a Product of Experts approach, combining two distinct learned distributions:

  • Quality Distribution: A Multi-Layer Perceptron (MLP) evaluates statistics (like distance to prototypes and classification entropy) to predict how useful an individual item is to the current classifier.
  • Diversity Distribution: To avoid redundancy, the model learns a feature space where it calculates the angle between items. It prefers items that are "orthogonal" (dissimilar) to those already selected for the batch.

By multiplying these two distributions, the model ensures it picks items that are both high-quality and diverse. Crucially, this selection process has linear complexity, making it much faster than previous batch-optimized methods.

Datasets

The researchers evaluated their model on two standard few-shot learning benchmarks:

  • miniImageNet: A subset of ImageNet with 100 classes, featuring $84 \times 84$ images.
  • CIFAR-100: A dataset of 100 classes with smaller $32 \times 32$ images.

In both cases, they split the classes so the model was tested on entirely new categories it hadn't seen during training, testing its ability to "learn how to learn" active learning.

Key Insights

  • Heuristics are Suboptimal for Batches: The paper demonstrates that while "Max-Entropy" is a classic go-to, it often performs worse than random selection in small batches because it lacks a mechanism to ensure diversity.
  • Learning to Ignore Noise: One of the most significant findings is the model's performance with distractors. Because the meta-learner is trained on episodes that include distractors, it learns to recognize the "statistical signature" of an irrelevant item and avoids picking it, whereas traditional heuristics are easily fooled.
  • Direct Optimization: Unlike heuristics which are "best guesses" at what might help, the meta-learner is trained to directly optimize the accuracy of the classifier on a query set after the selected batch is added.

Experimental Results

The meta-learning approach consistently outperformed Max-Entropy, Min-Max Similarity, and Random selection across all tests.

miniImageNet Results
CIFAR-100 Results
  • On miniImageNet (Batch size 20): The meta-learner improved accuracy by 18.1%, compared to 11.3% for Max-Entropy and 17.8% for Random selection.
  • Handling Distractors: In the most challenging setting (Batch size 20 with distractors, denoted as "w/ D"), the meta-learner achieved an 11.2% improvement on miniImageNet. In contrast, Max-Entropy only managed 2.8%, and Random selection reached 7.5%.
  • Consistency: The meta-learner was the only method that consistently beat the "Random" baseline across every single experimental configuration (batch sizes 5, 10, and 20) on both datasets.

Limitations and Future Work

While the results are promising, the authors note a few areas for growth:

  • Scaling: The experiments focused on few-shot learning (starting with very few labeled examples). It remains to be seen how well this meta-learning approach scales to "many-shot" scenarios with much larger support sets.
  • Interpretability: As with many RL-based meta-learners, it is difficult to pinpoint exactly why the model chooses one item over another beyond the broad "quality vs. diversity" framework.
  • Trade-offs: The authors suggest future work should look deeper into the dynamic trade-off between quality and diversity as the labeling process progresses.

Summary updated

BibTeX
@inproceedings{ravi2018meta,
  author = {Sachin Ravi and Hugo Larochelle},
  title = {{Meta-Learning for Batch Mode Active Learning}},
  booktitle = {Workshops @ ICLR},
  year = {2018},
  url = {https://openreview.net/forum?id=JQKQh1mTxM},
}
Figure