Learning Unambiguous Reduced Sequence Descriptions

Jürgen SchmidhuberNeurIPS 1991 · OpenReview

Introduction

Handling long-term dependencies has historically been one of the most significant challenges in training recurrent neural networks (RNNs). This paper introduces the "principle of history compression," a novel methodology that trains RNNs to manage long-term dependencies by systematically ignoring predictable data. By constructing a hierarchy of predictors where each level processes only "surprising" or unpredictable events, the system generates a compact, multi-scale representation of time. This approach dramatically accelerates learning on complex, extended sequences.

The "Long Time Lag" Problem

In the early 1990s, standard RNN training algorithms—such as Back-Propagation Through Time (BPTT) and Real-Time Recurrent Learning (RTRL)—faced a massive hurdle when attempting to link events separated by numerous time steps. The author identifies two fundamental flaws in these conventional approaches:

  1. Resource Waste: Standard RNNs process every single input at every time step. If an input is perfectly predictable and redundant, processing it wastes computational resources.
  2. Vanishing Information: As error signals propagate back through time step-by-step, the gradient information "washes out." This makes it nearly impossible for a conventional network to learn that an event occurring 20 steps in the past is responsible for a current error.

Core Methodology: History Compression

To overcome these limitations, the paper proposes History Compression, implemented via two primary architectural designs.

1. Self-Organizing Predictor Hierarchy

Rather than relying on a single, monolithic network, the system utilizes a stack of predictors ($P_0, P_1, \dots, P_n$).

  • The Base Predictor ($P_0$): Receives the raw input stream and attempts to predict the next step.
  • The "Surprise" Filter: If $P_0$ correctly predicts the next input, that input is deemed redundant and ignored. If it fails, the "unexpected" input—along with a timestamp—is passed up to the next level, $P_1$.
  • Hierarchical Abstraction: $P_1$ processes a compressed version of the history. This filtering continues up the chain. Because higher levels only receive "surprises," they operate on a slower, more abstract time scale where temporally distant events are brought structurally closer together.

2. Collapsing the Hierarchy (The 2-Net System)

To prevent the predictor stack from growing infinitely, the author introduces a collapsed, two-network system:

  • The Automatizer (A): Manages routine, predictable inputs.
  • The Chunker (C): Acts as an attentive observer, receiving only the inputs that the Automatizer fails to predict.
  • The Hand-off Mechanism: Crucially, the Automatizer is trained to predict both the next input and the internal state of the Chunker. Over time, the Automatizer absorbs the Chunker’s knowledge, rendering the Chunker obsolete for that specific sequence and freeing it to learn new abstractions.

3. Continuous History Compression

The paper also introduces a continuous variant of this system. Instead of a binary "yes/no" threshold for predictability, it uses a continuous value based on the output's probability distribution (entropy). This allows the network to make minor, proportional updates for nearly-expected inputs.

Key Insights

  • Surprise Equals Information: Drawing on classic Information Theory, the architecture assumes that predictable data carries no new information and thus requires no higher-level processing.
  • Temporal Abstraction: By filtering out the noise of predictable sequences, the network automatically discovers the underlying "macro-events" of a sequence.
  • Shortening the Credit Assignment Path: Skipping predictable steps physically shortens the distance between cause and effect in higher-level networks. This allows gradients to flow effectively across long time spans without vanishing.
  • Conscious vs. Automatic Learning: The dynamic between the Automatizer and Chunker mirrors human cognitive processes, where intense focus is required for new tasks (Chunker) until they become automatic habits (Automatizer).

Experimental Results

The history compression system was evaluated on two synthetic benchmarks designed specifically to break conventional RNNs, yielding stark improvements over the 1992 state-of-the-art:

  • Grammar Task: The network had to differentiate between strings generated by a grammar with local temporal structures and long, overlapping suffixes. While conventional algorithms failed completely, the chunking system solved the task within a few hundred thousand sequences.
  • 20-Step Lag Task: In a prediction task requiring information from exactly 20 time steps prior, conventional networks showed no improvement even after 1,000,000 training sequences. The 2-net chunking system solved the task in fewer than 5,000 sequences.
  • Computational Efficiency: Because higher-level networks are triggered infrequently, the chunking system required significantly less computation per time step than baseline models.

Limitations and Future Directions

Despite its success, the author notes several limitations:

  • Stability: There is no formal mathematical proof that the 2-net "collapsing" version is stable. The Automatizer risks "unlearning" old patterns while attempting to mimic the Chunker.
  • Hyperparameter Sensitivity: The discrete version of the system relies on an ad-hoc $\epsilon$ threshold to define what constitutes a prediction "mismatch."
  • Time Representation Complexity: While the paper suggests using a unique representation of time for higher-level networks, the optimal encoding of these timestamps for generalized tasks remains an open question for future research.

Summary updated

Abstract

Do you want your neural net algorithm to learn sequences? Do not lim(cid:173) it yourself to conventional gradient descent (or approximations thereof). Instead, use your sequence learning algorithm (any will do) to implement the following method for history compression. No matter what your fi(cid:173) nal goals are, train a network to predict its next input from the previous ones. Since only unpredictable inputs convey new information, ignore all predictable inputs but let all unexpected inputs (plus information about the time step at which they occurred) become inputs to a higher-level network of the same kind (working on a slower, self-adjusting time scale). Go on building a hierarchy of such networks. This principle reduces the descriptions of event sequences without 1088 of information, thus easing supervised or reinforcement learning tasks. Alternatively, you may use two recurrent networks to collapse a multi-level predictor hierarchy into a single recurrent net. Experiments show that systems based on these prin(cid:173) ciples can require less computation per time step and many fewer training sequences than conventional training algorithms for recurrent nets. Final(cid:173) ly you can modify the above method such that predictability is not defined in a yes-or-no fashion but in a continuous fashion.
BibTeX
@inproceedings{schmidhuber1991learning,
  author = {Jürgen Schmidhuber},
  title = {{Learning Unambiguous Reduced Sequence Descriptions}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1991},
  url = {https://openreview.net/forum?id=H1-UVvZ_WS},
}
Figure