RecMem: Recurrent Memory Compression for Long-Sequence Generative Recommendation

Haoyi Hu, Gaoyang Guo, Yunjia Xi, Wen Chen, Weiwen Liu, Jian Wu, Yuning Jiang, Weinan ZhangarXiv 2026 · arXiv

TL;DR

RecMem is a recurrent memory compression framework that allows generative recommender systems to process "lifelong" user histories with constant computational and memory costs. By compressing historical interactions into a fixed set of memory slots, it achieves a 14x speedup in inference latency compared to full-context models while retaining over 97% of their long-tail recommendation accuracy.

What Problem Does This Paper Address?

Modern recommender systems are shifting toward generative recommendation, treating a user's interaction history as a sequence-to-sequence task. While this approach captures complex user preferences, it hits a "context wall": Transformers have computational and memory costs that grow linearly (or quadratically) with the length of the input sequence.

For a user with thousands of interactions, feeding the entire history into a Transformer leads to context explosion, causing prohibitive latency and GPU memory consumption. Existing solutions usually involve "forgetting" old data (fixed-window models) or searching for relevant bits (retrieval methods), both of which can miss subtle, long-term patterns. The authors aim to create a memory mechanism that is space-invariant (fixed size), semantically faithful (retains useful info), and lifelong evolvable (remains stable over time).

Methods

The authors introduce RecMem, which processes user history in two stages: compression and prediction.

  1. Bifurcated Encoder-Decoder: RecMem uses a modified architecture based on the Hierarchical Sequential Transducer Unit (HSTU). It shares lower layers to capture general sequential dependencies but uses separate upper blocks for memory encoding and next-item decoding.
  2. Slot-wise Forget-Gated Updates: Instead of storing every item, RecMem compresses historical segments into a fixed number of memory slots ($M$). As new segments arrive, a gating mechanism (similar to a GRU or LSTM) decides what to keep from the old memory and what to incorporate from the new segment.
  3. Stream-Consistency Training: To ensure the model stays stable over long rollouts, the authors introduce a novel training objective. It aligns the predictions made when processing segments one-by-one (sequential) with those made when processing them all at once (packed). This prevents "approximation errors" from accumulating over thousands of updates.
Figure 1: Overview of RecMem. Top: U folds the K archived segments into a fixed-size memory state, one segment at a time. (a) In each update, the slot-wise forget gate is computed from the mask-aware mean of E(St). (b) Both branches start from the same MK−2, and the packed update uses repeated segment-local positions and a causal mask. The two final states are decoded with the same Xrec and candidates; sg denotes stop-gradient. (c) Recent tokens attend to all memory slots and causally to each other. The decoder outputs at all recent positions, Hrec, enter the NCE loss with shifted positives Y + and sampled negatives Y −; at inference, the last row of Hrec scores candidate items. Blue, green, and orange mark shared, encoder-side, and decoder-side components.

Datasets

The model was evaluated on MerRec, a large-scale benchmark dataset from the Mercari C2C marketplace. It contains interactions from over 5 million users, many with very long histories (up to 1,200 interactions in the training set and evaluated up to 10,000 interactions).

Key Insights and Ideas

  • The Fixed-Capacity Bottleneck: By forcing the model to use a fixed number of memory slots (e.g., 16 slots), the training process naturally turns the memory into a "predictive bottleneck." The encoder learns to discard noise and only preserve historical signals that are actually useful for predicting the next item.
  • Distant History Still Matters: A common assumption in RS is that only recent actions matter. RecMem's analysis shows that "archived-only" anchors (items the user interacted with long ago but not recently) still provide a significant predictive signal that RecMem successfully captures.
  • Stability via Theory: The authors provide a theoretical bound on "state discrepancy." They prove that under certain conditions (contraction and approximate semigroup consistency), the error in the memory state remains bounded regardless of how many thousands of segments are processed.

Key Results

  • Efficiency: RecMem uses only 16 memory slots and a 216-token decoder context, compared to 1,200 tokens for the baseline HSTU-1200. This results in a 7.65ms warm latency, roughly 14x faster than the 106.6ms of the full-context baseline.
  • Accuracy Retention: Despite the massive compression, RecMem-Stream retains 87.9% of Recall@10 and 97.0% of Recall@200 compared to the full-context model.
  • Memory Footprint: RecMem requires only 8 KiB of state per user, whereas segment-based methods like PHSTU require 432 KiB—a 54x reduction in storage.
  • Long-Term Stability: In an evaluation spanning 10,000 interactions for a fixed cohort of users, RecMem showed no signs of quality collapse or "capacity saturation," maintaining stable performance throughout.

Limitations and Future Work

  • ID-Based Focus: The current evaluation focuses on ID-based recommendations. The authors suggest that future work should explore richer textual or multimodal inputs.
  • Fixed Allocation: The number of memory slots and segment lengths are currently fixed. Adaptive allocation—giving more memory to "complex" users—could be a promising direction.
  • Theoretical Assumptions: The stability proof relies on specific assumptions like "contraction," which the authors approximate via spectral normalization but do not strictly certify.

Summary updated

Abstract

Generative recommendation benefits from long interaction histories, but the cost of processing them directly grows with history length. We propose RecMem, a recurrent memory compression framework guided by three desiderata: space invariance, semantic fidelity, and lifelong evolvability. RecMem compresses historical segments into a fixed set of memory slots using a bifurcated encoder-decoder with shared lower layers and slot-wise forget-gated updates. Recommendation supervision trains the memory to preserve predictive information, while stream-consistency training aligns predictions from sequential and packed processing. By construction, the memory state and decoder context are independent of history length; under contraction and approximate semigroup consistency, we further derive a horizon-independent bound on the state discrepancy between these two processing orders. On MerRec, RecMem uses 16 memory slots and a 216-token decoder context, compared with 1,200 tokens for HSTU-1200, while retaining 87.9% of HSTU-1200's Recall@10 and 97.0% of its Recall@200. It outperforms PHSTU and the evaluated recent-only and retrieval baselines, and analyses of distant history show that archived interactions contribute predictive signal.
BibTeX
@misc{hu2026recmem,
  author = {Haoyi Hu and Gaoyang Guo and Yunjia Xi and Wen Chen and Weiwen Liu and Jian Wu and Yuning Jiang and Weinan Zhang},
  title = {{RecMem: Recurrent Memory Compression for Long-Sequence Generative Recommendation}},
  year = {2026},
  eprint = {2611.17921},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17921},
}
Figure