Constant Step Sizes for Non-Stationary Tracking

This direction is split into other directions: constant step-size adaptation. Its last version stays readable and citable here.

Version 1 of 186 papers125 conceptsfrozen 2026-10-04

In classical optimization, the goal is convergence. We typically employ a decaying learning rate—starting large to explore the landscape and gradually shrinking to zero to settle precisely into a global minimum. This "vanishing" step size is mathematically necessary to eliminate gradient noise and guarantee that an algorithm stops moving once it finds the truth.

But what if the "truth" is a moving target? In real-world applications like financial forecasting, autonomous soaring, or tracking moving signal sources, the underlying data distribution is non-stationary. If an algorithm’s learning rate decays to zero, it eventually loses its "plasticity"—the ability to adapt to new information. It becomes a frozen model in a changing world.

A broad body of research suggests a counter-intuitive fix: to track a dynamic world, we must abandon the quest for absolute convergence and instead embrace constant step sizes.

The Plasticity Dilemma

The fundamental trade-off in non-stationary environments is between tracking error and gradient noise. As Jianshu Chen and Zaid J. Towfic demonstrate in their work on diffusion adaptation, a constant step size prevents the update magnitude from vanishing. This ensures the filter remains "alert," allowing it to pivot when data statistics shift.

When we use a decaying learning rate, the model's sensitivity to new data decreases over time. Rupam Mahmood identifies this as a "loss of plasticity," a failure mode where neural networks become too rigid to learn new tasks. By maintaining a non-zero learning rate, we effectively implement a "forgetting factor." The algorithm prioritizes recent observations over the distant past, enabling it to follow a moving optimal point within a bounded neighborhood of error.

Optimization as Continuous Filtering

If we view optimization not as a search for a static point but as a filtering process, the role of the constant step size becomes clearer. Gianluca M. Bencomo and Thomas L. Griffiths argue in Implicit Maximum a Posteriori Filtering that standard training routines are essentially state-tracking mechanisms.

In this framework, the step size acts as a gain that balances our trust in the current model versus new, noisy evidence. Pei Jung Chung and Alfred O. Hero applied this intuition to tracking multiple moving sources, showing that a constant step size allows recursive algorithms to cross source paths and maintain high accuracy at a fraction of the computational cost of traditional search-based methods.

Collective Intelligence and Diffusion

This principle extends naturally to decentralized networks. In Diffusion Adaptation over Networks, Ali H. Sayed explains that for a network of agents to collectively track a target, they must avoid the "halting" problem of decaying step sizes.

By using constant step sizes, agents can share "beliefs" or gradients with neighbors to reach a consensus that is both stable and agile. Xiuheng Wang and Ricardo Augusto Borsoi have recently extended this to Riemannian manifolds, proving that constant step sizes allow agents to track evolving distributions even on complex, curved geometries like those found in distributed PCA or Gaussian Mixture Models.

Beyond Constants: Increasing the Rate

Perhaps the most provocative finding in this area is that a constant rate might sometimes be too slow. Matthew Fahrbach and Vahab Mirrokni show that in the presence of significant distribution shifts, the optimal strategy may be to increase the learning rate.

When the "velocity" of the environment's change increases, the model needs more "speed" to catch up. Their adaptive schedules allow the learning rate to plateau or rise in proportion to the detected shift, ensuring the model doesn't just lag behind the moving target but actively pursues it.

The Gradient Equilibrium

Traditional "no-regret" frameworks often struggle with non-stationarity because they are designed to compete with the best fixed strategy. A newer perspective, Gradient Equilibrium (GEQ), introduced by Anastasios N. Angelopoulos and Michael I. Jordan, shifts the focus.

Instead of minimizing regret, GEQ aims to keep the average of gradients along a sequence of predictions near zero. As they demonstrate in Gradient Equilibrium in Online Learning, simple constant step sizes are sufficient to achieve this equilibrium. This ensures that even as the environment drifts, the model remains "unbiased" in the long run, providing calibrated uncertainty and robust performance without the need for complex, decaying schedules.

By prioritizing responsiveness over asymptotic convergence, these researchers have shown that "never stopping" is the only way to keep up with a world that never stays still.

Papers in this version

Showing 60 of 86 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_constant_step_sizes_for_non_stationary_t_3146f0a2,
  title = {Constant Step Sizes for Non-Stationary Tracking},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/constant-step-sizes-for-non-stationary-tracking/rdr_3146f0a27eb54f21a2bb57a1e8669136},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_constant_step_sizes_for_non_stationary_t_3146f0a2_v1,
  title = {Constant Step Sizes for Non-Stationary Tracking},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/constant-step-sizes-for-non-stationary-tracking/rdr_3146f0a27eb54f21a2bb57a1e8669136/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}