A Psychology-based Unified Dynamic Framework for Curriculum Learning

Guangyu Meng, Qingkai Zeng, John P. Lalor, Hong YuarXiv 2024 · DOI · arXiv

Curriculum Learning (CL) mimics the human educational process by presenting training data in an order of increasing difficulty. While conceptually simple, implementing effective CL in machine learning faces two primary hurdles: objectively measuring data difficulty (Difficulty Measurement, DM) and determining an optimal training pace (Training Scheduler, TS). Traditional methods often rely on static heuristics or computationally expensive reinforcement learning.

Bridging Psychometrics and Machine Learning

This paper introduces PUDF (Psychology-based Unified Dynamic Framework), which leverages Item Response Theory (IRT) to create a unified metric for both data difficulty and model ability. Unlike previous approaches that treat DM and TS as separate problems, PUDF uses a single "language" to align the curriculum with the learner's current state.

Comparison of PUDF with existing Curriculum Learning categories. PUDF is unique in providing both automatic difficulty measurement and scheduling with low computational complexity.

The PUDF Methodology

The framework consists of two main components:

1. IRT-AC: Difficulty Measurement via Artificial Crowds

To avoid the cost of human labeling, the authors employ an Artificial Crowd (AC)—a collection of diverse, pre-trained Large Language Models (LLMs). By observing which models correctly process specific examples, the framework applies IRT to calculate a global difficulty score ($b_i$) for each item. In this model, difficulty is defined as the ability level required for a 50% success probability.

Illustration of Item Response Theory (IRT) curves. The difficulty parameter $b_i$ shifts the probability curve along the ability axis ($\theta_j$), allowing for a standardized measurement of item hardness.

2. DDS-MAE: Dynamic Ability-Based Scheduling

During the fine-tuning phase, the Dynamic Data Selector (DDS-MAE) periodically estimates the "student" model's current ability ($\theta$) using a small data sample. It then filters the training set to include only examples where the difficulty is within the model's reach ($b \le \theta$). This dynamic adjustment prevents the model from being overwhelmed by noise or excessively complex data, even allowing the curriculum to "slow down" if the model's performance dips.

Key Results and Insights

The authors evaluated PUDF across various benchmarks, including GLUE, AG News, MedQA, and GSM8K, using models like Llama 3.1-8B and DeBERTaV3.

  • Efficiency Gains: On the AG News dataset (1M examples), PUDF reduced training time for Llama 3.1-8B by nearly 70% compared to standard fine-tuning.
  • Improved Accuracy: On the specialized MedQA dataset, the framework achieved a 6.65% relative improvement in accuracy.
  • Data Pruning: A significant finding was that models often reach peak performance using only 70–95% of the data. By excluding the most difficult (and potentially noisy) examples, PUDF avoids the "distraction" that can occur in the final stages of standard training.

Conclusion

PUDF represents a significant step forward in making Curriculum Learning practical for large-scale model training. By grounding the framework in established psychometric principles, the authors provide a robust, model-independent method for data selection that is both computationally efficient and highly effective across classification and generative tasks. Future work may further refine the "zone of proximal development" to determine if showing models slightly more challenging data can further accelerate learning.

Summary updated

BibTeX
@misc{meng2024psychology,
  author = {Guangyu Meng and Qingkai Zeng and John P. Lalor and Hong Yu},
  title = {{A Psychology-based Unified Dynamic Framework for Curriculum Learning}},
  year = {2024},
  eprint = {2408.05326},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.05326},
}
Figure