Introduction
In the late 1980s and early 90s, training recurrent neural networks (RNNs) was notoriously difficult. The standard approach, Backpropagation Through Time (BPTT), frequently struggled with "vanishing gradients" or became trapped in local minima when attempting to learn dependencies over long durations.
Richard Rohwer's "Moving Targets" algorithm flips the script on training RNNs by treating hidden node activations as adjustable variables rather than just intermediate calculations. By optimizing these "moving targets" in state space, the algorithm can solve difficult temporal problems—such as responding to a signal from 100 time steps in the past—that often baffle standard backpropagation techniques.
The Core Philosophy: State Space vs. Weight Space
In a standard setup, we only have "targets" for the output nodes. The hidden nodes are left to figure themselves out via the chain rule. This paper explores a different philosophy: what if we treat hidden nodes as if they had their own training data, but let that data "move" and evolve as the network learns?
The most profound idea here is that optimization might be more effective in the "state space" (the activations) than the "weight space." By focusing on what the nodes should be doing, the weights almost take care of themselves. The author credits Paul Werbos with the qualitative idea that networks should set intermediate goals; this algorithm provides the formal math to make those goals computationally flexible.
Methodology
The "Moving Targets" method introduces a clever mathematical decoupling:
- Hidden States as Variables: Every hidden node at every time step is assigned a variable activation value. These are the "moving targets."
- The Activation Deficit: Instead of a standard output error, the author defines an error function based on how well the current weights and previous states predict the next state (for both hidden and target nodes).
- Linear Optimization for Weights: Because the error function is quadratic with respect to the weights (when the targets are fixed), the optimal weights for a given set of targets can be found by solving a system of linear equations. This is much faster and more stable than gradient descent on weights.
- Gradient Descent on Targets: The actual "learning" happens by calculating the gradient of the error with respect to the hidden state variables. The algorithm "moves" these targets to better satisfy both the desired output and the network's internal dynamics.
- Curvature Avoidance: The paper also introduces a specialized minimization algorithm that estimates the diagonal of the Hessian (curvature) and steers the optimization toward "flatter" regions of the error surface to avoid narrow, difficult canyons.
Experimental Tasks and Results
The paper evaluates the algorithm on three synthetic "toy" problems designed to test dynamical complexity:
- Limit Cycle Switching: A network must switch between four different oscillation frequencies based on binary input signals.
- Temporal Parity Detection: A classic logic problem extended over time, where the network must track whether it has seen an even or odd number of pulses.
- 100-Step Delay Task: A benchmark where the network must provide a specific response to a trigger that occurred 100 discrete time steps in the past.
Key Achievements
- Solving the 100-Step Delay: The algorithm successfully trained a network to remember a signal across 100 time steps. At the time, this was a significant achievement, as standard BPTT rarely functioned beyond 10-20 steps without extreme tuning. By treating each time step's activation as an independent variable initially, the algorithm avoids the recursive "multiplication of gradients" that causes BPTT to fail over long time horizons.
- Quartic Error Functions: The author found that using a quartic (power of 4) error function worked better than the standard squared error. This "punishes" large outliers more severely, forcing the network to fix its biggest mistakes first.
- Temporal Weighting: Success was improved by weighting errors more heavily at later time steps, essentially teaching the network to get the "end result" right and then work backward to figure out the necessary preceding states.
Limitations and Future Work
While powerful, the method has notable constraints:
- Scalability: The primary drawback is the sheer number of variables. Since every hidden node at every time step is a variable, the memory and computational requirements grow linearly with the length of the training sequences.
- Computational Cost: Solving the linear system for weights and managing the large target gradient vector makes this method more "heavyweight" per iteration than standard backpropagation.
- Local Minima: The author notes that the algorithm is still susceptible to local minima, especially in complex problems where the "moving targets" might settle into a sub-optimal configuration.
Ultimately, while the method's computational demands currently limit it to smaller "toy" problems, it opens a promising door for discovering distant temporal correlations that other methods simply miss.
Summary updated