Abstract
This research bridges cognitive science and machine learning by demonstrating that "active learning" principles—specifically peer-to-peer interaction—outperform "passive learning" in both humans and neural networks. By simulating human-like discussion through bidirectional knowledge distillation between two small student models, the authors establish a training framework that consistently outperforms traditional passive knowledge transfer from a large teacher model.
Introduction
In human education, passive learning (e.g., listening to a lecture) is an efficient delivery method but often falls short of maximizing student achievement compared to active learning (e.g., discussing, synthesizing). In machine learning, "active learning" typically refers to data-labeling strategies. This paper redefines the term by focusing on the cognitive process of learning. Drawing from the Interactive-Constructive-Active-Passive (ICAP) framework, the authors investigate whether simulating human-like interactive discussion between models yields better performance than standard, passive knowledge transfer.
Methods
The researchers conducted parallel experiments to compare learning paradigms in humans and machines.
Experiment 1: Human Learning
54 undergraduates were divided into two groups:
- Passive (L group): Students watched a lecture and studied alone for 36 minutes.
- Active (D group): Students studied alone for 18 minutes followed by 18 minutes of peer discussion.
Experiment 2: Machine Learning
The authors utilized Knowledge Distillation (KD) to simulate these learning styles in text classification models (TextCNN and LSTM):
- Passive ML: Standard KD where a large, pre-trained "Teacher" model ($M_T$) transfers knowledge to a smaller "Student" model ($M_S$).
- Active ML: Bidirectional KD where two small "Student" models ($M_S$) are trained simultaneously, exchanging knowledge ($M_S \leftrightarrow M_S$).
Key Results
The study found that active learning paradigms significantly improved performance in both domains.
Human Outcomes
The active discussion group (D group) vastly outperformed the passive group (L group) across all test types, including verbatim recall, paraphrasing, and knowledge transfer. The total mean score for the active group was 38.41, compared to 27.52 for the passive group ($p < .001$).
Machine Outcomes
The active learning framework ($M_S \leftrightarrow M_S$) outperformed the passive framework ($M_T \rightarrow M_S$) across almost all datasets (DBpedia, YahooAnswers, AGNews, Yelp, IMDB) for both CNN and LSTM architectures. For instance, on the AGNews dataset with a CNN, the active approach achieved 90.21% accuracy compared to 89.68% for the passive approach.
Key Insights
- Interaction over Absorption: In humans, splitting time between individual study and peer discussion is more effective than continuous passive absorption.
- Peer-to-Peer Efficacy: In machines, two "novice" models exchanging information can outperform a novice model passively receiving information from an "expert" model.
- Cognitive Blueprints for AI: Psychological frameworks like ICAP can be successfully translated into loss functions and training loops to improve neural network performance.
Conclusion
This work demonstrates that cognitive science offers a valid architectural blueprint for machine learning. By moving beyond simple data-labeling strategies and focusing on the learning process itself, researchers can design more efficient and effective neural network architectures. Future work will continue to explore cognitive process-based approaches to further bridge the gap between human and machine intelligence.
Summary updated