When an optimization algorithm solves a problem from scratch, it performs a "cold start." Lacking any prior knowledge, it must initialize its parameters randomly or at a generic origin, navigating a vast search space to locate a minimum. However, in many real-world applications—such as robotics, video processing, or iterative machine learning training—we do not solve isolated problems. Instead, we solve a sequence of highly correlated problems over time.
Because physical systems and model parameters evolve continuously, the solution to the problem at time $t$ is an exceptionally accurate approximation of the solution at time $t+1$. By seeding the solver with the previous step's solution—a strategy known as temporal warm-starting—we bypass the global search phase, collapsing the required number of iterations and unlocking real-time performance.
Shifting Horizons in Model Predictive Control
In trajectory planning and Model Predictive Control (MPC), warm-starting takes a structured form. An MPC controller computes an optimal sequence of future actions over a time horizon. At the next time step, rather than discarding this sequence, the controller shifts the previous plan forward, discards the executed action, and appends a heuristic guess for the final state.
Szilárd Novoth et al. demonstrate this in Distributed Formation Control for Multi-Vehicle Systems With Splitting and Merging Capability, showing that initializing distributed optimization solvers with shifted results from previous steps reduces the required iterations for convergence by nearly two-thirds. This computational saving is vital for real-time safety. For instance, Sophie Hall et al. utilize warm-starting in Real-time Projected Gradient-based Nonlinear Model Predictive Control with an Application to Anesthesia Control to achieve rapid feedback in dynamic medical environments. Similarly, Andreas B. Martinsen shows in Autonomous docking using direct optimal control that warm-starting reduces nonlinear optimization solver times by nearly an order of magnitude.
Crucially, this efficiency directly impacts system stability. In real-time deployments, a solver may be forced to terminate early due to strict timing constraints. Anusha Srikanthan et al. prove in Closed-loop Analysis of ADMM-based Suboptimal Linear Model Predictive Control that warm-starting guarantees closed-loop stability and feasibility even when the underlying solver is stopped early after a fixed, finite number of iterations.
Bypassing the Denoising Bottleneck in Diffusion Models
Generative diffusion models have revolutionized robotics and pose estimation, but their iterative denoising process is notoriously slow, typically requiring dozens of sequential forward passes starting from pure Gaussian noise. Temporal warm-starting fundamentally alters this dynamic by replacing the random initial state with a structured guess.
Instead of starting from scratch, the generation process adds a small amount of noise to the trajectory or pose calculated in the previous time step. Hao Ma et al. leverage this in Constraint-Aware Diffusion Guidance for Robotics: Real-Time Obstacle Avoidance for Autonomous Racing, demonstrating that this approach drastically reduces the number of denoising iterations required while ensuring temporal consistency across control outputs.
In the context of robotic manipulation, Yufei Duan et al. introduce the Real-Time Iteration Scheme for Diffusion Policy, shifting the previous plan forward to achieve up to a 30$\times$ speedup in inference time without retraining the model. For 3D perception, Jiyao Zhang et al. apply this concept in Generative Category-level Object Pose Estimation via Diffusion Models, utilizing the output from previous temporal steps to shrink the computational search space and enable real-time object tracking.
Reusing Internal States and Dual Variables in Machine Learning
The utility of temporal warm-starting extends beyond physical trajectories into the internal optimization loops of machine learning architectures and training routines. When model parameters or inputs change incrementally, the internal states of the network can be preserved to accelerate convergence.
For example, Federico Danieli et al. propose DeepPCR: Parallelizing Sequential Operations in Neural Networks, where activations calculated in a previous training iteration serve as the initial guess for a parallel solver in the current step, exploiting the slow evolution of model parameters. In online learning scenarios, Darius Masoum Zadeh-Jousdani leverages temporal correlations in streaming data in Efficient Online Learning with Predictive Coding Networks: Exploiting Temporal Correlations, using converged hidden states from the previous timestep to halve the inference iterations needed for the current frame.
Furthermore, warm-starting can be applied to dual optimization variables. In structured prediction, Ofer Meshi et al. show in Learning Efficiently with Approximate Inference via Dual Losses that storing dual variables for each training example across epochs provides a high-quality initialization for subsequent inference steps, significantly reducing the computational burden of gradient estimation as the model weights stabilize.
Exploiting Geometric and Physical Continuity
In physical simulations and geometric optimization, objects move continuously through space, meaning their spatial relationships change incrementally between frames. Accelerated geometric solvers can exploit this continuity directly.
Louis Montaut et al. demonstrate this in GJK++: Leveraging Acceleration Methods for Faster Collision Detection, where the distance solution from a preceding simulation frame initializes the current distance calculation, minimizing the iterations required for convergence in dynamic environments. Similarly, Tingrui Zhang et al. apply this to continuous trajectory optimization in Continuous Implicit SDF Based Any-shape Robot Trajectory Optimization, initializing a gradient-based search for the closest obstacle using the optimal time step from the previous iteration.