Human-Like Active Learning: Machines Simulating the Human Learning Process

Jaeseo Lim, Hwiyeol Jo, Byoung-Tak Zhang, Jooyong ParkarXiv 2020 · arXiv

Abstract

This research bridges cognitive science and machine learning by demonstrating that "active learning" principles—specifically peer-to-peer interaction—outperform "passive learning" in both humans and neural networks. By simulating human-like discussion through bidirectional knowledge distillation between two small student models, the authors establish a training framework that consistently outperforms traditional passive knowledge transfer from a large teacher model.

Introduction

In human education, passive learning (e.g., listening to a lecture) is an efficient delivery method but often falls short of maximizing student achievement compared to active learning (e.g., discussing, synthesizing). In machine learning, "active learning" typically refers to data-labeling strategies. This paper redefines the term by focusing on the cognitive process of learning. Drawing from the Interactive-Constructive-Active-Passive (ICAP) framework, the authors investigate whether simulating human-like interactive discussion between models yields better performance than standard, passive knowledge transfer.

Methods

The researchers conducted parallel experiments to compare learning paradigms in humans and machines.

Experiment 1: Human Learning

54 undergraduates were divided into two groups:

  • Passive (L group): Students watched a lecture and studied alone for 36 minutes.
  • Active (D group): Students studied alone for 18 minutes followed by 18 minutes of peer discussion.

Experiment 2: Machine Learning

The authors utilized Knowledge Distillation (KD) to simulate these learning styles in text classification models (TextCNN and LSTM):

  • Passive ML: Standard KD where a large, pre-trained "Teacher" model ($M_T$) transfers knowledge to a smaller "Student" model ($M_S$).
  • Active ML: Bidirectional KD where two small "Student" models ($M_S$) are trained simultaneously, exchanging knowledge ($M_S \leftrightarrow M_S$).
Comparison of training paradigms: (a) standard training, (b) passive knowledge distillation, and (c) active peer-to-peer knowledge distillation.

Key Results

The study found that active learning paradigms significantly improved performance in both domains.

Human Outcomes

The active discussion group (D group) vastly outperformed the passive group (L group) across all test types, including verbatim recall, paraphrasing, and knowledge transfer. The total mean score for the active group was 38.41, compared to 27.52 for the passive group ($p < .001$).

Human performance comparison between L group (passive) and D group (active) across multiple cognitive metrics.

Machine Outcomes

The active learning framework ($M_S \leftrightarrow M_S$) outperformed the passive framework ($M_T \rightarrow M_S$) across almost all datasets (DBpedia, YahooAnswers, AGNews, Yelp, IMDB) for both CNN and LSTM architectures. For instance, on the AGNews dataset with a CNN, the active approach achieved 90.21% accuracy compared to 89.68% for the passive approach.

Key Insights

  • Interaction over Absorption: In humans, splitting time between individual study and peer discussion is more effective than continuous passive absorption.
  • Peer-to-Peer Efficacy: In machines, two "novice" models exchanging information can outperform a novice model passively receiving information from an "expert" model.
  • Cognitive Blueprints for AI: Psychological frameworks like ICAP can be successfully translated into loss functions and training loops to improve neural network performance.

Conclusion

This work demonstrates that cognitive science offers a valid architectural blueprint for machine learning. By moving beyond simple data-labeling strategies and focusing on the learning process itself, researchers can design more efficient and effective neural network architectures. Future work will continue to explore cognitive process-based approaches to further bridge the gap between human and machine intelligence.

Summary updated

Abstract

Although the use of active learning to increase learners' engagement has recently been introduced in a variety of methods, empirical experiments are lacking. In this study, we attempted to align two experiments in order to (1) make a hypothesis for machine and (2) empirically confirm the effect of active learning on learning. In Experiment 1, we compared the effect of a passive form of learning to active form of learning. The results showed that active learning had a greater learning outcomes than passive learning. In the machine experiment based on the human result, we imitated the human active learning as a form of knowledge distillation. The active learning framework performed better than the passive learning framework. In the end, we showed not only that we can make build better machine training framework through the human experiment result, but also empirically confirm the result of human experiment through imitated machine experiments; human-like active learning have crucial effect on learning performance.
BibTeX
@misc{lim2020human,
  author = {Jaeseo Lim and Hwiyeol Jo and Byoung-Tak Zhang and Jooyong Park},
  title = {{Human-Like Active Learning: Machines Simulating the Human Learning Process}},
  year = {2020},
  eprint = {2011.03733},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2011.03733},
}
Figure