Few Is Enough: Task-Augmented Active Meta-learning for Brain Cell Classification

Pengyu Yuan, Aryan Mobiny, Jahandar Jahanipour, Xiaoyang Li, Pietro Antonio Cicalese, Badrinath Roysam, Vishal M. Patel, Maric Dragan, Hien Van Nguyen

TL;DR

This paper introduces AGILE, a novel meta-learning framework that significantly improves adaptation to new tasks with minimal data. By augmenting existing tasks and actively selecting informative samples using Bayesian uncertainty, AGILE achieves state-of-the-art performance in brain cell classification with just 1% of the training data and a single gradient update.

What Problem Does This Paper Address?

Deep Neural Networks (DNNs) struggle when faced with distribution shifts in data, a common issue in real-world applications like medical imaging. Retraining models from scratch is computationally expensive. Meta-learning offers a solution by enabling models to adapt quickly to new tasks. However, traditional meta-learning methods require a large number of diverse tasks for training and often select training samples randomly, which can be suboptimal, especially when data is scarce. This paper addresses the challenge of adapting DNNs to new, unseen tasks efficiently, using very few labeled examples and a limited number of training tasks, a scenario particularly relevant for specialized domains like brain cell classification.

Methods

The proposed AGILE framework consists of two main phases:

  • Task-Augmented Meta-Learning: This phase focuses on generating a robust initial model that is sensitive to data distribution changes. It leverages a meta-learning algorithm, specifically inspired by Model-Agnostic Meta-Learning (MAML), but enhances it with a novel task augmentation technique. This technique artificially increases the diversity of meta-training tasks by applying transformations to existing tasks. These transformations include:
  • Flipping labels.
  • Shuffling input channels (representing different biomarkers).
  • Rotating images. This augmentation helps the meta-learner extract more useful features and improve its ability to adapt to new tasks.

  • Active Learning with Bayesian Uncertainty: In the second phase, when adapting to a new "real" task with limited data, AGILE employs active learning guided by Bayesian dropout uncertainty estimates. Instead of randomly selecting samples, the model uses dropout during inference to approximate Bayesian inference and identify samples where its prediction is most uncertain. These "hard" samples are then prioritized for training, leading to more efficient learning and better performance, especially when the number of available training samples is extremely small.

Datasets

The experiments were conducted using a dataset of 4000 brain cell images from rat brain tissue sections, representing 5 major cell types: neurons, astrocytes, oligodendrocytes, microglia, and endothelial cells. Seven biomarkers were used as feature channels. For each cell type, a binary classification task was created (identifying if a cell belongs to that type or not). The dataset was split such that 60% of cells were used as potential training samples and the remaining 40% for testing. The paper uses a subset of these cells (1% and 10%) for evaluating the few-shot learning capabilities of AGILE.

Key Insights and Ideas

  • Task Augmentation is Crucial for Limited Meta-Tasks: The paper highlights that when the number of meta-training tasks is limited, standard meta-learning can struggle. Artificially increasing task diversity through augmentation significantly improves the meta-learner's ability to generalize to unseen tasks.
  • Active Learning with Uncertainty is Key for Extreme Few-Shot Scenarios: For scenarios with very few training samples (e.g., 1% of data), randomly selecting samples is highly inefficient. Prioritizing samples with high prediction uncertainty (identified via Bayesian dropout) allows the model to learn more effectively and achieve substantial performance gains.
  • Single Gradient Step Adaptation is Achievable: AGILE demonstrates that with its task-augmented meta-learning and active learning strategy, a model can achieve high accuracy on a new task after just one gradient update, a significant improvement over traditional methods.
  • Practicality of Imbalanced Sampling: Unlike some meta-learning approaches that assume class-balanced sampling, AGILE's active learning strategy naturally handles imbalanced sample selection, prioritizing difficult classes, which is more aligned with real-world data acquisition where challenging cases often require more attention.

Key Results

  • AGILE achieved 90% accuracy on a new brain cell type classification task using only 1% of the training data and a single gradient update. This represents a 50 percentage point improvement over a state-of-the-art meta-learning algorithm (MAML).
  • With 10% of the training data, AGILE reached 95% accuracy, matching the performance of a model trained on 60% of the data (the upper bound).
  • The task-augmented meta-learning phase (AGILE phase I) showed a significant improvement in precision, recall, and F1-score compared to vanilla meta-learning and transfer learning methods when using limited data.
  • The active learning component (AGILE phase II) further boosted performance, especially at very low training sample regimes.
  • Traditional transfer learning and MAML methods, when applied with limited data, performed worse than a simple vanilla model trained from scratch, indicating they were "stuck in the local minimum."

Limitations and Future Work

  • Limitations: The paper acknowledges that while AGILE's active learning strategy can handle imbalanced sample selection, the meta-training itself still uses class-balanced samples. The effectiveness of the task augmentation techniques might be domain-specific.
  • Future Work: The authors suggest that AGILE's framework can be applied to other diagnostic systems or detection algorithms that deal with diverse input data and require rapid adaptation. Further exploration into more sophisticated task augmentation strategies and active learning criteria could also be beneficial.

Summary updated

Abstract

Deep Neural Networks (or DNNs) must constantly cope with distribution changes in the input data when the task of interest or the data collection protocol changes. Retraining a network from scratch to combat this issue poses a significant cost. Meta-learning aims to deliver an adaptive model that is sensitive to these underlying distribution changes, but requires many tasks during the meta-training process. In this paper, we propose a tAsk-auGmented actIve meta-LEarning (AGILE) method to efficiently adapt DNNs to new tasks by using a small number of training examples. AGILE combines a meta-learning algorithm with a novel task augmentation technique which we use to generate an initial adaptive model. It then uses Bayesian dropout uncertainty estimates to actively select the most difficult samples when updating the model to a new task. This allows AGILE to learn with fewer tasks and a few informative samples, achieving high performance with a limited dataset. We perform our experiments using the brain cell classification task and compare the results to a plain meta-learning model trained from scratch. We show that the proposed task-augmented meta-learning framework can learn to classify new cell types after a single gradient step with a limited number of training samples. We show that active learning with Bayesian uncertainty can further improve the performance when the number of training samples is extremely small. Using only 1% of the training data and a single update step, we achieved 90% accuracy on the new cell type classification task, a 50% points improvement over a state-of-the-art meta-learning algorithm.
BibTeX
@inproceedings{yuan2020few,
  author = {Pengyu Yuan and Aryan Mobiny and Jahandar Jahanipour and Xiaoyang Li and Pietro Antonio Cicalese and Badrinath Roysam and Vishal M. Patel and Maric Dragan and Hien Van Nguyen},
  title = {{Few Is Enough: Task-Augmented Active Meta-learning for Brain Cell Classification}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  url = {https://openreview.net/forum?id=o2wW1ltVoS6w},
}
Figure