LiveTune: Dynamic Parameter Tuning for Training Deep Neural Networks

Soheil Zibakhsh Shabgahi, Nojan Sheybani, Aiden Tabrizi, Farinaz KoushanfararXiv 2023 · DOI · arXiv

Introduction: The "Stop-and-Go" Bottleneck

In modern machine learning workflows, hyperparameter tuning is often a fragmented, inefficient process. When a researcher identifies a suboptimal learning rate or a flawed reward function, the standard procedure requires halting execution, modifying the source code, and restarting the training process. This "stop-and-go" cycle introduces significant non-productive overhead.

Comparison of the traditional restart-based workflow versus the LiveTune dynamic update approach.

This restart is far from instantaneous. It involves reloading massive libraries (e.g., PyTorch, TensorFlow), re-initializing model weights, and re-transferring data from disk to RAM and finally to the GPU. LiveTune addresses this inefficiency by enabling real-time, dynamic parameter injection into a running process without requiring a program restart.

System Architecture: LiveVariables and Triggers

LiveTune replaces static constants with specialized objects called LiveVariables. These variables are initialized with a unique tag and a designated TCP port. Each variable operates a dedicated listener thread that monitors its port for updates. When a new value is transmitted via the LiveTune API, the variable updates its state using semaphores to ensure thread safety and prevent race conditions within the main training loop.

Beyond simple value updates, the framework introduces LiveTriggers. These are dynamic boolean flags that allow developers to execute specific procedures—such as resetting an optimizer or saving a checkpoint—on demand. A centralized "Dictionary Thread" manages these variables across the system, ensuring consistency and providing a unified interface for the user. The framework is designed to be framework-agnostic, requiring minimal code changes to integrate into existing PyTorch or TensorFlow loops.

Quantifying the "Warmup Tax"

The researchers evaluated the cost of program restarts across several standard benchmarks, including ImageNet-1k, TinyImageNet, and CIFAR100, using various VGG architectures (VGG11 through VGG19).

Warmup time in seconds for various models and datasets, illustrating the significant time lost during program initialization.

As shown in the evaluation, the "warmup" phase—setting up the GPU and transferring data—can consume up to 60 seconds and 5.4 kJ of energy per restart. In a typical day of iterative experimentation, these costs compound into hours of lost productivity and a substantial carbon footprint. By eliminating the need for restarts, LiveTune recovers this time and energy for every parameter adjustment.

Human-in-the-Loop Reinforcement Learning

One of the most compelling applications of LiveTune is in Reinforcement Learning (RL). In the "Hungry Thirsty Domain" (a 4x4 grid-world), agents often struggle with competing objectives. LiveTune allows researchers to "shape" the reward function dynamically. By observing a real-time heatmap of the agent's position, a user can instantly tweak rewards for specific states to nudge the agent toward the goal. This intervention led to a 5x improvement in fitness (86.51 vs. 5.08) compared to static baselines.

Experimental Results and User Study

In an in-person competition using the Fashion-MNIST dataset, participants using LiveTune achieved 85% accuracy, outperforming the control group (83%). The primary driver of this success was iteration speed: LiveTune users ran an average of 33 more epochs within a 1.5-hour window because they were not penalized by restart delays. This suggests that reducing the cost of experimentation directly encourages more frequent and effective exploration of the hyperparameter space.

Conclusion and Future Directions

LiveTune demonstrates that removing the friction of program restarts significantly enhances both computational efficiency and human productivity. While the current implementation focuses on manual intervention via a web interface, future work aims to integrate fully automated, unsupervised feedback loops. Additionally, the logic of LiveVariables could extend beyond machine learning to other engineering simulations that suffer from high initialization overhead, such as Computer-Aided Design (CAD) optimization.

Summary updated

Abstract

Feedback-driven optimization, such as traditional machine learning training, is a static process that lacks real-time adaptability of hyperparameters. Tuning solutions for optimization require trial and error paired with checkpointing and schedulers, in many cases feedback from the algorithm is overlooked. Adjusting hyperparameters during optimization usually requires the program to be restarted, wasting utilization and time, while placing unnecessary strain on memory and processors. We present LiveTune, a novel framework allowing real-time parameter adjustment of optimization loops through LiveVariables. Live Variables allow for continuous feedback-driven optimization by storing parameters on designated ports on the system, allowing them to be dynamically adjusted. Extensive evaluations of our framework on standard machine learning training pipelines show saving up to 60 seconds and 5.4 Kilojoules of energy per hyperparameter change. We also show the feasibility and value of LiveTune in a reinforcement learning application where the users change the dynamics of the reward structure while the agent is learning showing 5x improvement over the baseline. Finally, we outline a fully automated workflow to provide end-to-end, unsupervised feedback-driven optimization.
BibTeX
@misc{shabgahi2023livetune,
  author = {Soheil Zibakhsh Shabgahi and Nojan Sheybani and Aiden Tabrizi and Farinaz Koushanfar},
  title = {{LiveTune: Dynamic Parameter Tuning for Training Deep Neural Networks}},
  year = {2023},
  eprint = {2311.17279},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.17279},
}
Figure