ScreenerNet: Learning Curriculum for Neural Networks

Tae-Hoon Kim, Jonghyun Choi

Abstract

Traditional curriculum learning often relies on hand-crafted rules or computationally expensive memory buffers to prioritize training data. We present ScreenerNet, an attachable neural network that automatically learns a training curriculum by weighting samples based on their current significance. By jointly training this "screener" alongside a main model, we achieve faster convergence and higher final accuracy across reinforcement learning and computer vision tasks without human-defined rules or sampling bias.

Introduction

In human education, learning follows a structured progression from simple to complex concepts. This principle, known as Curriculum Learning, has long been sought in machine learning to accelerate training. However, existing approaches face significant challenges. Most rely on hand-crafted heuristics—such as training on low-loss samples first—which may not be optimal for deep architectures. Furthermore, "hard" sampling methods can introduce bias; if a model rejects a difficult sample early, it may never learn to handle edge cases. ScreenerNet addresses these issues by learning a "soft" curriculum that dynamically weights samples without discarding data.

Methods: The ScreenerNet Architecture

ScreenerNet is a secondary, typically smaller, neural network that acts as a "syllabus designer." Unlike previous methods that require tracking the historical loss of every training sample, ScreenerNet makes weighting decisions based on the current state of the model and the input sample itself.

The joint training process of ScreenerNet and the Main Network.

As shown in the architecture above, the system employs a joint training strategy:

  1. Soft Weighting: ScreenerNet assigns a weight $w_x \in [0, 1]$ to each sample. This ensures no data is entirely ignored, mitigating sampling bias while focusing the main network's capacity on significant samples.
  2. Saddle-like Loss Function: The screener is trained using a novel loss function designed to assign high weights to samples where the main network exhibits high error and low weights where the error is minimal.
  3. Plug-and-Play Integration: ScreenerNet can be attached to existing models and combined with other techniques like Prioritized Experience Replay (PER) to further refine data prioritization.

Experimental Results

The effectiveness of ScreenerNet was evaluated across diverse benchmarks, including Reinforcement Learning (Cart-pole) and Supervised Visual Recognition (MNIST, CIFAR10, and Pascal VOC 2012).

Performance comparison across Cart-pole, Pascal VOC 2012, CIFAR10, and MNIST datasets.

Key results include:

  • Faster Convergence: Across all datasets, models equipped with ScreenerNet reached higher accuracy or reward levels significantly faster than baseline models.
  • Synergy with RL: In the Cart-pole task, ScreenerNet outperformed standard Prioritized Experience Replay (PER). Combining both methods yielded the best results, suggesting that soft weighting provides a complementary benefit to hard sampling.
  • Handling Ambiguity: Qualitative analysis on MNIST revealed that ScreenerNet prioritizes ambiguous or blurry digits (e.g., a '4' that resembles a '9'), forcing the model to master difficult edge cases.

Discussion and Future Work

ScreenerNet demonstrates that a learned curriculum can outperform human-designed rules. Interestingly, the authors found that sharing weights between the main network and ScreenerNet degraded performance, suggesting that the task of evaluating sample significance is fundamentally distinct from classification or control.

The architecture of the screener itself is a critical hyperparameter; a screener with complexity similar to the main network generally yields the best performance. While joint training is effective, future work could explore "momentum" in weight updates to improve stability. Additionally, the concept of "Growing a Brain"—progressively adding layers to the screener as tasks become more complex—remains a promising avenue for scaling this approach to even larger datasets.

Summary updated

Abstract

We propose to learn a curriculum or a syllabus for supervised learning and deep reinforcement learning with deep neural networks by an attachable deep neural network, called ScreenerNet. Specifically, we learn a weight for each sample by jointly training the ScreenerNet and the main network in an end-to-end self-paced fashion. The ScreenerNet neither has sampling bias nor requires to remember the past learning history. We show the networks augmented with the ScreenerNet achieve early convergence with better accuracy than the state-of-the-art curricular learning methods in extensive experiments using three popular vision datasets such as MNIST, CIFAR10 and Pascal VOC2012, and a Cart-pole task using Deep Q-learning. Moreover, the ScreenerNet can extend other curriculum learning methods such as Prioritized Experience Replay (PER) for further accuracy improvement.
BibTeX
@misc{kim2018screenernet,
  author = {Tae-Hoon Kim and Jonghyun Choi},
  title = {{ScreenerNet: Learning Curriculum for Neural Networks}},
  year = {2018},
  eprint = {1801.00904},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1801.00904},
}
Figure