Understanding Recurrent Neural State Using Memory Signatures

Skanda Koppula, Khe Chai Sim, Kean K. Chin

Introduction

Recurrent Neural Networks (RNNs), specifically Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures, have become foundational in sequence modeling tasks such as language modeling and Automatic Speech Recognition (ASR). Despite their success, the internal representations of these models remain largely opaque. Unlike traditional n-gram models, which rely on explicit historical counts, RNNs encode information into continuous hidden states. This paper introduces "memory signatures," a post-hoc probing technique designed to quantify and visualize the historical information retained within these hidden states.

Methodology: Probing the Hidden State

The core of the proposed method involves training auxiliary decoders to extract past inputs from the current hidden state of a frozen, pre-trained RNN. For a given layer $l$ and a specific time delay $\delta$, a small fully-connected network is trained to predict the input token that occurred $\delta$ steps in the past using only the recurrent state at the current time step $t$.

By aggregating the accuracy of these decoders across various delays and layers, the authors construct a "memory signature." This signature acts as a diagnostic tool, revealing the network's capacity to retain specific types of information over time without modifying the primary model's weights or training objective.

Key Insights

The Impact of Model Capacity

One of the most direct findings is the relationship between the size of the recurrent state and its memory capacity. As the state size increases, the model's ability to accurately recall past inputs improves significantly.

Memory recall accuracy for GRU (a) and LSTM (b) across different state sizes (columns) and time delays (rows).

As shown in the figure above, increasing the state size from 80 to 640 units leads to a substantial boost in decodable history. This increased memory capacity correlates with lower test perplexity, suggesting that the model's performance is tied to its ability to maintain a high-fidelity representation of the recent past.

Layer-wise Information Abstraction

In language models, memory signatures reveal a distinct trend: information tends to "fade" or become more abstracted in deeper layers. While the first layer maintains a very high-fidelity record of recent raw inputs, subsequent layers show a decrease in the accuracy of exact token recall.

LSTM memory signatures across four layers, showing a decrease in exact input recall in deeper layers as information is abstracted.

This suggests that as information propagates upward through the network, the model filters out raw surface-level details in favor of more abstract, higher-level representations. Interestingly, the paper notes that this trend is reversed in ASR encoder models, where deeper layers accumulate a wider window of historical context to support the attention mechanism.

Domain Adaptation and Model Complementarity

Memory signatures also provide a visual explanation for why certain techniques work. For instance, when a model trained on Shakespeare is fine-tuned on Wall Street Journal (WSJ) data, its memory signature shifts to match the WSJ-trained model's profile. This indicates that fine-tuning fundamentally reorganizes how the model prioritizes historical information.

Furthermore, the signatures show that LSTMs and traditional n-gram models are "good at remembering" different things. LSTMs might excel at long-range dependencies, while n-grams capture local statistical patterns more effectively. This complementarity explains the performance gains often seen when ensembling these two types of models.

Conclusion

Memory signatures offer a powerful new lens for interpreting the internal dynamics of RNNs. By transforming opaque hidden states into interpretable heatmaps of historical recall, researchers can better understand model limitations, debug performance issues, and gain insights into the mechanisms of domain adaptation and layer-wise abstraction. Future work may extend this technique to more complex input types, such as words or continuous audio features, further demystifying the "black box" of recurrent neural architectures.

Summary updated

BibTeX
@inproceedings{koppula2018understanding,
  author = {Skanda Koppula and Khe Chai Sim and Kean K. Chin},
  title = {{Understanding Recurrent Neural State Using Memory Signatures}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2018},
  doi = {10.1109/icassp.2018.8462652},
}
Figure