Learning to Rehearse in Long Sequence Memorization

Zhu Zhang, Chang Zhou, Jianxin Ma, Zhijie Lin, Jingren Zhou, Hongxia Yang, Zhou Zhao

TL;DR

Rehearsal Memory (RM) is a novel approach to long-sequence reasoning that mimics human cognitive processes to prevent "catastrophic forgetting" in neural networks. By combining a "history sampler" that identifies important past events with self-supervised "rehearsal" tasks, the model learns to prioritize and retain critical information within a fixed-size memory. This approach outperforms existing memory-augmented networks on complex text, video, and recommendation tasks.

What Problem Does This Paper Address?

Most AI reasoning tasks assume the model can always access the entire raw input (like a full book or a long video). However, for real-world applications like online recommendation systems or long-form video analysis, storing and re-processing massive sequences is computationally expensive and slow.

Memory-Augmented Neural Networks (MANNs) try to solve this by compressing inputs into a fixed-size "write-read" memory. But MANNs face two major hurdles:

  1. Gradual Forgetting: As new information comes in, older information is continuously overwritten and lost.
  2. Information Blindness: Standard MANNs treat all input data as equally important, often wasting limited memory slots on "noise" while forgetting the "signal" needed to answer a future query.

The Rehearsal Memory (RM) Framework

The authors propose the Rehearsal Memory (RM) framework, which introduces a segment-level memory machine and a dual-training strategy to combat these issues.

1. Rehearsal Memory Machine (RMM)

Instead of processing data token-by-token, the RMM breaks sequences into segments. It uses a Transformer encoder to capture local context and a gate-based update to write these features into memory slots.

The overall framework of Rehearsal Memory, detailing the Rehearsal Memory Machine and the dual-training strategy involving self-supervised rehearsal.

2. How to Rehearse: Self-Supervised Tasks

To combat forgetting, the model "rehearses" past fragments using two self-supervised tasks:

  • Recollection: The model takes a past fragment, masks 50% of its items, and must reconstruct them using only its current internal memory.
  • Familiarity: The model must distinguish between a real fragment from its history and a "fake" one sampled from elsewhere.

3. What to Rehearse: The History Sampler

To solve the "importance" problem, the authors use a History Sampler. This acts as a "teacher" model that has temporary access to the full raw input during training. It calculates which fragments are most relevant to a specific query. The "student" (the RMM) then focuses its rehearsal efforts on these high-importance fragments, learning to prioritize task-relevant information.

Datasets and Key Insights

The model was evaluated across four diverse domains:

  • bAbI: A synthetic reasoning benchmark for short-sequence QA.
  • NarrativeQA: Long-sequence text QA based on book summaries (avg. 600+ tokens).
  • ActivityNet-QA: Long-term video QA (avg. video length of 180 seconds).
  • XLong: A lifelong sequence recommendation dataset consisting of Alibaba user click logs (sequence length of 1,000).

Key takeaways from the architecture include: * Segment-Level over Item-Level: Processing data in chunks (segments) allows the model to capture bidirectional context within each chunk, making the initial encoding much more robust than traditional RNN-based MANNs. * Privileged Training: The use of a "History Sampler" is a clever application of knowledge distillation. By letting a teacher model "peek" at the full data during training, the memory-constrained student learns better heuristics for what is worth keeping. * High Masking for Memory Strength: While models like BERT use a 15% masking ratio for pre-training, RM uses 50%. This high ratio forces the model to rely heavily on its long-term memory slots rather than just guessing based on the surrounding unmasked items in a fragment.

Key Results

RM consistently outperformed state-of-the-art memory models like the Differentiable Neural Computer (DNC) and the Compressive Transformer (CT).

Performance comparison of RM against baselines (STM, CT) across different memory slot numbers on NarrativeQA and ActivityNet-QA.
  • Accuracy: On the ActivityNet-QA video task, RM achieved 36.3% accuracy, surpassing the Compressive Transformer (35.4%) and significantly beating the DNC (30.3%).
  • Recommendation: On the XLong dataset, RM achieved an AUC of 0.8817, outperforming specialized recommendation models like MIMN (0.8731).
  • Efficiency: RM maintains a fixed memory size regardless of input length, providing $O(1)$ storage complexity, which is critical for scaling to "lifelong" sequences.

Limitations and Future Work

While RM shows significant improvements, the authors note that the "History Sampler" requires a separate training phase with access to raw inputs, which might not be feasible in all real-time learning scenarios. Additionally, while the model alleviates forgetting, the fixed memory size still acts as a fundamental bottleneck; determining the optimal number of memory slots ($K$) for different task complexities remains an open area for exploration. Future work may involve more dynamic memory allocation or exploring different rehearsal strategies beyond recollection and familiarity.

Summary updated

Abstract

Existing reasoning tasks often have an important assumption that the input contents can be always accessed while reasoning, requiring unlimited storage resources and suffering from severe time dela...
BibTeX
@inproceedings{zhang2021learning,
  author = {Zhu Zhang and Chang Zhou and Jianxin Ma and Zhijie Lin and Jingren Zhou and Hongxia Yang and Zhou Zhao},
  title = {{Learning to Rehearse in Long Sequence Memorization}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=LTJ6fCqbmI},
}
Figure