Interactive Training: Feedback-Driven Neural Network Optimization

Wentao Zhang, Yang Young Lu, Yuntian Deng

Introduction: From the Oven to the Stove

Traditional neural network training follows a "static" paradigm. Practitioners define hyperparameters, such as learning rate schedules and data augmentation strategies, before execution. Once the process begins, they often passively monitor logs. If training instabilities arise—such as loss spikes or vanishing gradients—the standard recourse is to terminate the job, adjust settings, and restart from a checkpoint or from scratch. This "set-and-forget" approach, akin to baking in a closed oven, is increasingly inefficient as models grow in scale.

Conceptual comparison between static "oven" training and interactive "stove" training.

The authors propose Interactive Training, a framework that transforms this process into a dynamic, real-time conversation. By allowing human experts or AI agents to intervene mid-stream, training becomes more like cooking on a stove, where the "heat" can be adjusted the moment a problem is detected.

The Interactive Training Framework

The proposed system is built upon the Hugging Face Trainer class and comprises three integrated components:

  1. Control Server: A FastAPI-based mediator that facilitates two-way communication. It receives real-time metrics from the training engine and broadcasts intervention commands from the user.
  2. Interactive Trainer: The core engine utilizing custom callbacks to poll for commands at every gradient step. It can dynamically update optimizer parameters (e.g., learning rate, weight decay), swap datasets, or revert to previous checkpoints without halting the training loop.
  3. Frontend Dashboard: A React-based interface providing real-time visualizations of loss curves and gradient norms. Unlike passive monitoring tools, it includes a control panel for direct intervention.

A standout feature is the support for branching trajectories. This allows researchers to revert to a stable checkpoint and launch parallel training branches to empirically compare different intervention strategies side-by-side.

Experimental Validation

The framework was evaluated through two primary case studies:

  • Language Modeling (Wikitext-2): GPT-2 was fine-tuned to demonstrate how human-in-the-loop and AI-agent-driven adjustments can stabilize training.
  • NeuralOS Simulation: A diffusion-based operating system simulator was trained using 746 real-user interaction sequences. This case study highlighted the framework's ability to incorporate "live" data collected during deployment, allowing the model to adapt to actual usage patterns in real-time.

Results and Analysis

The experiments demonstrated that interactive interventions lead to superior outcomes compared to static baselines.

Validation loss curves and learning rate schedules comparing static training to interactive human-in-the-loop training.

As shown in Figure 6, a human expert monitoring GPT-2 fine-tuning could manually reduce the learning rate in response to loss oscillations. This intervention resulted in a lower final validation loss than the standard linear decay recipe. Furthermore, the authors tested an "AI Pilot" model using an LLM (OpenAI’s o4-mini) to monitor logs. When the agent detected instability, it autonomously lowered the learning rate, successfully "rescuing" runs that would have otherwise failed due to excessively high initial learning rates.

In the NeuralOS study, the ability to inject real-time user data significantly improved the model's qualitative performance on complex tasks, such as browser navigation, which were poorly handled by models trained solely on synthetic data.

Discussion and Future Work

While Interactive Training offers significant gains in efficiency and performance, it introduces challenges regarding reproducibility. Because interventions are dynamic, the specific path to a trained model depends on the "pilot's" expertise. The authors mitigate this by logging all interventions for "replay" and auditability.

Future research will focus on developing specialized "Intervention Agents"—AI models trained specifically to detect and correct training anomalies—and identifying new "Training Health Metrics" to provide earlier warning signals for both human and automated controllers.

Summary updated

BibTeX
@inproceedings{zhang2025interactive,
  author = {Wentao Zhang and Yang Young Lu and Yuntian Deng},
  title = {{Interactive Training: Feedback-Driven Neural Network Optimization}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure