DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback

Zaid Khan, Elias Stengel-Eskin, Jaemin Cho, Mohit Bansal

In the world of machine learning, the spotlight typically shines on the "student"—the model being trained. However, an educator's effectiveness is fundamentally tied to the quality of the curriculum. Currently, developing such a curriculum is a labor-intensive, manual process where researchers analyze model errors and brainstorm new data to address them. DataEnvGym, a framework developed by researchers at UNC Chapel Hill, automates this loop by creating autonomous "teacher" agents.

The Data Bottleneck and Autonomous Teaching

While Large Language Models (LLMs) can generate synthetic data, the process remains largely human-steered. Researchers must interpret evaluation results and decide on the next learning objectives. DataEnvGym addresses this by providing a standardized testbed for developing agents that act as teachers. These agents iteratively evaluate a student model, plan a data generation strategy, generate examples, and train the student.

The DataEnvGym cycle: an iterative loop between an evaluation environment and a data-generation agent.

The DataEnvGym Framework

DataEnvGym treats data generation as a sequential decision-making task, akin to Reinforcement Learning. The core loop involves an Agent (the teacher) and an Environment (the student's training and evaluation rig). The framework supports three environment types with varying structural complexity:

  • Open-Ended: The most flexible mode. The teacher receives a raw list of student errors and must independently determine the next data generation steps.
  • Skill-List: An LLM-based "skill discovery" module categorizes tasks into human-interpretable skills. The teacher monitors performance metrics for each skill to target specific gaps.
  • Skill-Tree: The most structured approach. Skills are organized into a hierarchy (a "skill forest"). The teacher chooses between Explore (discovering new sub-skills) and Exploit (generating more data for known weak skills).

Experimental Evaluation

The framework was tested across four diverse domains: Visual Question Answering (GQA, NaturalBench), Mathematics (MATH), Programming (LiveCodeBench), and Tool-Use (MnMs). Student models like PaliGemma and Llama-3 were fine-tuned using data generated by teacher agents (e.g., GPT-4o).

Key Results

The autonomous teachers consistently improved student performance across all benchmarks. Notably, agents that utilized feedback (the "With State" condition) significantly outperformed those that generated data without observing the student's specific errors.

Performance improvements across different environments and tasks, highlighting the benefit of state-aware data generation.

As shown in the results, the Open-Ended environment with state feedback improved performance by an average of 4.60% across tasks. In multimodal VQA tasks, the structured Skill-Tree environment proved particularly effective, suggesting that complex domains benefit from hierarchical curriculum scaffolding.

Insights and Future Directions

A critical finding is the importance of the "Zone of Proximal Development." Data generation is most effective when it targets skills that are neither too easy (already mastered) nor too difficult (beyond current learning capacity). Furthermore, the quality of the teacher agent is paramount; replacing GPT-4o with a less capable model led to a marked decrease in student gains.

While DataEnvGym currently focuses on Supervised Fine-Tuning (SFT), future iterations could incorporate Reinforcement Learning from Human Feedback (RLHF) or richer feedback mechanisms, such as student confidence scores or chain-of-thought rationales. Despite the computational costs associated with the full generation-training-evaluation loop, DataEnvGym represents a significant step toward fully autonomous AI development.

Summary updated

BibTeX
@inproceedings{khan2025dataenvgym,
  author = {Zaid Khan and Elias Stengel-Eskin and Jaemin Cho and Mohit Bansal},
  title = {{DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=00SnKBGTsz},
}
Figure