Active Curriculum Learning

Borna Jafarpour, Dawn Sepehr, Nick PogrebnyakovACL 2021

TL;DR

This paper introduces Active Curriculum Learning (ACL), a novel method that combines Active Learning (AL) and Curriculum Learning (CL) to improve the efficiency of training machine learning models. By leveraging both the dynamic informativeness of AL and the structured learning of CL, ACL demonstrates superior performance in Named Entity Recognition (NER) tasks, suggesting a more intelligent way to select and present data for annotation.

What Problem Does This Paper Address?

Modern deep learning models thrive on vast amounts of labeled data, but acquiring these labels is often expensive and time-consuming. Active Learning (AL) aims to mitigate this by intelligently selecting the most "informative" unlabeled data points for annotation, thereby reducing the overall labeling effort. Simultaneously, Curriculum Learning (CL) draws inspiration from human learning, proposing that models learn more effectively when data is presented in a structured, "easy-to-hard" sequence.

While both AL and CL have proven effective individually, this paper explores their synergistic potential. The core challenge addressed is how to best combine these two paradigms: can the dynamic, model-driven selection of AL be enhanced by the structured, expert-designed progression of CL? The authors investigate the inherent relationship between AL and CL, hypothesizing that AL might implicitly follow a curriculum, and propose a unified framework, ACL, to explicitly leverage both.

Methods

The paper's primary contribution is the Active Curriculum Learning (ACL) framework. ACL integrates AL and CL by creating a combined score that balances the informativeness of a data point (as determined by the current model, the AL component) with its difficulty (as defined by a predefined curriculum, the CL component).

Specifically, ACL uses a linear combination of AL and CL scores:

$\psi_{ACL}(s, M_i) = \alpha \frac{Y_{AL}(s, M_i)}{\max|Y_{AL}(s, M_i)|} + \beta \frac{Y_{CL}(s)}{max(Y_{CL}(s))}$

Here, $Y_{AL}(s, M_i)$ represents the informativeness score of sentence $s$ from model $M_i$ at step $i$, and $Y_{CL}(s)$ is the curriculum score for sentence $s$. The parameters $\alpha$ and $\beta$ control the trade-off between AL and CL. The algorithm then selects the $k$ examples with the best combined score for annotation.

The authors also introduce seven novel curricula for NLP tasks, going beyond traditional metrics like sentence length and word frequency. These new curricula leverage linguistic features such as part-of-speech tags, dependency relations, and even scores from pre-trained language models like GPT-2 and Longformer.

Datasets

The experiments were conducted on two publicly available English-language Named Entity Recognition (NER) datasets:

  • OntoNotes5
  • CoNLL 2003

These datasets were used for training and evaluating the proposed ACL framework.

Key Insights and Ideas

  • AL implicitly follows a curriculum: The paper's initial investigation reveals that AL strategies, even without explicit curriculum design, tend to select data points that exhibit curriculum-like properties (e.g., min-margin AL selects "easier" sentences based on GPT score). This suggests a natural synergy between the two.
  • Novel curricula are effective: The introduction of linguistically-informed curricula, particularly those derived from deep language models (GPT_SCORE and LL_LOSS), significantly contributes to the performance gains observed with ACL.
  • Combining AL and CL is beneficial: The proposed ACL framework consistently outperforms standalone AL strategies, demonstrating that explicitly integrating expert-designed curricula with model-driven active selection leads to more efficient learning.
  • Trade-off between AL and CL is important: The parameters $\alpha$ and $\beta$ in ACL allow for tuning the influence of AL and CL, suggesting that the optimal balance can be dataset and task-dependent.

Key Results

The experimental results, particularly the Mean Normalized Difference (MND) metric, show that AL strategies do diverge from random selection in ways that align with curriculum properties.

Table 2 summarizes the performance of ACL. For instance, on the OntoNotes5 dataset with a min-margin AL strategy, using the GPT_SCORE curriculum with $\alpha=1$ and $\beta=0.5$ achieved an F1 score of 0.4, outperforming the baseline AL strategy ($\alpha=1, \beta=0$) which achieved 0.23. Similar improvements were observed across various curriculum and AL strategy combinations on both datasets, highlighting the utility of ACL. The paper notes that the top 5 ACL combinations consistently outperformed the baseline AL strategy.

Limitations and Future Work

The authors acknowledge that the current ACL framework uses a fixed linear combination of AL and CL scores ($\alpha$ and $\beta$). They suggest that future work could explore more dynamic or model-based approaches to combining these scores.

Further research directions include:

  • Investigating the detailed mechanisms by which CL aids AL.
  • Exploring model-based techniques for combining AL and CL, moving beyond fixed weights.
  • Conducting experiments with other NLP tasks and potentially using multiple curricula simultaneously.
  • Evaluating the novel curricula in an isolated CL setting.

Summary updated

BibTeX
@inproceedings{jafarpour2021active,
  author = {Borna Jafarpour and Dawn Sepehr and Nick Pogrebnyakov},
  title = {{Active Curriculum Learning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure