Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performance

Haitham Bou-Ammar, Antoine Grosnit, Alexandre Maraval, S N Refinath, Zichao Zhao, J. Doran, Giuseppe Paolo, Albert Thomas, Juan M. González, Abhineet Kumar, Khyati Khandelwal, Abdelhakim Benechehab, Hamza Cherkaoui, Youssef Attia El-Hili, Kun Shao, Jianye Hao, Jun Yao, Balázs Kégl, Jun WangResearch Square 2025 · DOI

Abstract

Human expertise emerges through iterative cycles of interaction, reflection, and internal model updating, which are central to cognitive theories such as Kolb’s experiential learning and Vygotsky’s zone of proximal development. In contrast, current AI systems, particularly large language models (LLMs) agents, rely on static pretraining or rigid workflows and lack mechanisms for continual adaptation. Recent studies have identified early cognitive traits in LLM agents, including reflection, revision, and self-correction, which suggest foundational elements of human-like experiential learning. This leads to a key question: Can we design LLM agents capable of structured, cognitively grounded learning similar to human processes? To address this, we propose a computational framework of Kolb’s learning cycle with Vygotsky’s ZPD for autonomous agents. Our architecture separates extrinsic functions (environment interaction) from intrinsic functions (internal reflection and abstraction), enabling cognitively grounded scaffolded learning, where the agent initially learns within structured, supportive environments, followed by open-ended generalisation. This approach empowers agents to master complex, many-step tasks; domains that traditional fine-tuning or simple reflective methods could not tackle effectively. Its potential is powerfully demonstrated through direct competition with humans in real-world Kaggle data science challenges. Learning fully automated, end-to-end data science code generation across 81 tasks, our system, Agent K, demonstrated the ability to perform the entire workflow without human intervention, achieving an Elo-MMR score of 1694, placing it beyond median performance of the Kaggle Masters (the top 2% among over 200,000 users) included in our study. With 9 gold, 8 silver, and 12 bronze medals level performance – including 4 gold and 4 silver on prize-awarding competitions – Agent K is the first AI system to successfully integrate Kolb- and Vygotsky-inspired human cognitive learning, marking a major step toward generalist AI.

Introduction

While Large Language Models (LLMs) have shown remarkable capabilities, current autonomous agents often rely on static pre-training or rigid, one-shot workflows. These systems frequently struggle with complex, multi-step tasks like data science, which require iterative refinement, reflection on errors, and the ability to adapt strategies over time. In contrast, human experts learn through a continuous cycle of doing, reflecting, and abstracting. This research addresses a fundamental gap: designing AI agents that learn through structured cycles of experience and adaptation, mirroring human cognitive processes.

Methods: Cognitively Grounded Learning

The proposed framework, embodied in Agent K, is built upon two pillars of educational psychology:

1. Kolb’s Experiential Learning Cycle

Agent K's architecture explicitly separates extrinsic functions (interacting with the environment, such as executing code) from intrinsic functions (internal reflection and abstraction). Rather than simply generating code, the agent analyzes failures, updates its internal plan, and iterates.

The computational framework of Kolb’s experiential learning cycle for autonomous agents.

2. Vygotsky’s Zone of Proximal Development (ZPD)

The agent undergoes "scaffolded learning," where it is initially guided through structured sub-tasks. This includes a Workspace Scaffold for data preparation and a Solution Scaffold for modeling. As the agent masters these components, the scaffolds are removed, allowing it to use its internalized knowledge to tackle open-ended problems autonomously.

The scaffolding learning process within the Agent's Zone of Proximal Development (ZPD).

Agent K utilizes LLMs (primarily Qwen2.5-72B) to drive these functions, building a "tree of solutions" where each node represents a full data science pipeline. It evaluates feedback from validation scores or error logs to decide whether to refine a current approach or explore a new strategy.

Evaluation on Real-World Data Science

The system was evaluated on a benchmark of 81 real-world Kaggle competitions. Unlike synthetic benchmarks, Kaggle presents messy data, diverse file structures, and high risks of overfitting. The tasks spanned Tabular Data (55%), Computer Vision (24%), Natural Language Processing (10%), and Multimodal tasks (11%).

Key Results and Insights

  • Human-Level Performance: Agent K achieved an Elo-MMR score of 1694, surpassing the median performance of Kaggle Masters (the top 2% of over 200,000 users).
  • Medal Achievement: In "featured" prize-awarding competitions, it earned the equivalent of 4 gold and 4 silver medals. Across the full benchmark, it secured 29 medal-equivalent rankings (9 gold, 8 silver, 12 bronze).
  • Internalization of Knowledge: A key insight is that the agent "internalizes" its scaffolded experience. It summarizes successful reasoning into "Scaffold-CoTs" (Chain-of-Thought traces), which serve as mental models for future unguided tasks.
  • Efficiency: By using ZPD-inspired abstractions, Agent K achieved a 5% performance improvement while requiring 2x less exploration time compared to agents without these cognitive structures.

Discussion and Limitations

While Agent K demonstrates state-of-the-art performance, some limitations exist. Testing on historical competitions meant the agent had access to modern architectures (e.g., Vision Transformers) not available to original human competitors. However, "time-consistent" technology tests still yielded 19 medals. Additionally, while the authors used recent 2024/2025 competitions to mitigate LLM memorization risks, Agent K still performed in the top 12%. Future work aims to extend this framework to more creative reasoning tasks, such as the ARC-AGI benchmark.

Summary updated

Abstract

<title>Abstract</title> Human expertise emerges through iterative cycles of interaction, reflection, and internal model updating, which are central to cognitive theories such as Kolb’s experiential learning and Vygotsky’s zone of proximal development. In contrast, current AI systems, particularly large language models (LLMs) agents, rely on static pretraining or rigid workflows and lack mechanisms for continual adaptation. Recent studies have identified early cognitive traits in LLM agents, including reflection, revision, and self-correction, which suggest foundational elements of human-like experiential learning. This leads to a key question: Canwe design LLM agents capable of structured, cognitively grounded learning similar to human processes? To address this,we propose a computational framework ofKolb’s learning cycle with Vygotsky’s ZPD for autonomous agents. Our architecture separates extrinsic functions (environmentinteraction) fromintrinsic functions (internal reflection and abstraction), enabling cognitively grounded scaffolded learning, where the agent initially learns within structured, supportive environments, followed by open-ended generalisation. This approach empowers agents to master complex, many-step tasks ; domains that traditional fine-tuning or simple reflective methods could not tackle effectively. Its potential is powerfully demonstrated through direct competition with humans in real-world Kaggle data science challenges. Learning fully automated, end-to-end data science code generation across 81 tasks, our system, Agent K, demonstrated the ability to perform the entire workflow without human intervention, achieving an Elo-MMR score of 1694, placing it beyond median performance of the Kaggle Masters (the top 2% among over 200,000 users) included in our study. With 9 gold, 8 silver, and 12 bronze medals level performance – including 4 gold and 4 silver on prize-awarding competitions – Agent K is the first AI system to successfully integrate Kolb- and Vygotsky-inspired human cognitive learning, marking a major step toward generalist AI.
BibTeX
@inproceedings{bouammar2025kolb,
  author = {Haitham Bou-Ammar and Antoine Grosnit and Alexandre Maraval and S N Refinath and Zichao Zhao and J. Doran and Giuseppe Paolo and Albert Thomas and Juan M. González and Abhineet Kumar and Khyati Khandelwal and Abdelhakim Benechehab and Hamza Cherkaoui and Youssef Attia El-Hili and Kun Shao and Jianye Hao and Jun Yao and Balázs Kégl and Jun Wang},
  title = {{Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performance}},
  booktitle = {Research Square},
  year = {2025},
  doi = {10.21203/rs.3.rs-7472642/v1},
}
Figure