TL;DR
RecMem is a recurrent memory compression framework that allows generative recommender systems to process "lifelong" user histories with constant computational and memory costs. By compressing historical interactions into a fixed set of memory slots, it achieves a 14x speedup in inference latency compared to full-context models while retaining over 97% of their long-tail recommendation accuracy.
What Problem Does This Paper Address?
Modern recommender systems are shifting toward generative recommendation, treating a user's interaction history as a sequence-to-sequence task. While this approach captures complex user preferences, it hits a "context wall": Transformers have computational and memory costs that grow linearly (or quadratically) with the length of the input sequence.
For a user with thousands of interactions, feeding the entire history into a Transformer leads to context explosion, causing prohibitive latency and GPU memory consumption. Existing solutions usually involve "forgetting" old data (fixed-window models) or searching for relevant bits (retrieval methods), both of which can miss subtle, long-term patterns. The authors aim to create a memory mechanism that is space-invariant (fixed size), semantically faithful (retains useful info), and lifelong evolvable (remains stable over time).
Methods
The authors introduce RecMem, which processes user history in two stages: compression and prediction.
- Bifurcated Encoder-Decoder: RecMem uses a modified architecture based on the Hierarchical Sequential Transducer Unit (HSTU). It shares lower layers to capture general sequential dependencies but uses separate upper blocks for memory encoding and next-item decoding.
- Slot-wise Forget-Gated Updates: Instead of storing every item, RecMem compresses historical segments into a fixed number of memory slots ($M$). As new segments arrive, a gating mechanism (similar to a GRU or LSTM) decides what to keep from the old memory and what to incorporate from the new segment.
- Stream-Consistency Training: To ensure the model stays stable over long rollouts, the authors introduce a novel training objective. It aligns the predictions made when processing segments one-by-one (sequential) with those made when processing them all at once (packed). This prevents "approximation errors" from accumulating over thousands of updates.
Datasets
The model was evaluated on MerRec, a large-scale benchmark dataset from the Mercari C2C marketplace. It contains interactions from over 5 million users, many with very long histories (up to 1,200 interactions in the training set and evaluated up to 10,000 interactions).
Key Insights and Ideas
- The Fixed-Capacity Bottleneck: By forcing the model to use a fixed number of memory slots (e.g., 16 slots), the training process naturally turns the memory into a "predictive bottleneck." The encoder learns to discard noise and only preserve historical signals that are actually useful for predicting the next item.
- Distant History Still Matters: A common assumption in RS is that only recent actions matter. RecMem's analysis shows that "archived-only" anchors (items the user interacted with long ago but not recently) still provide a significant predictive signal that RecMem successfully captures.
- Stability via Theory: The authors provide a theoretical bound on "state discrepancy." They prove that under certain conditions (contraction and approximate semigroup consistency), the error in the memory state remains bounded regardless of how many thousands of segments are processed.
Key Results
- Efficiency: RecMem uses only 16 memory slots and a 216-token decoder context, compared to 1,200 tokens for the baseline HSTU-1200. This results in a 7.65ms warm latency, roughly 14x faster than the 106.6ms of the full-context baseline.
- Accuracy Retention: Despite the massive compression, RecMem-Stream retains 87.9% of Recall@10 and 97.0% of Recall@200 compared to the full-context model.
- Memory Footprint: RecMem requires only 8 KiB of state per user, whereas segment-based methods like PHSTU require 432 KiB—a 54x reduction in storage.
- Long-Term Stability: In an evaluation spanning 10,000 interactions for a fixed cohort of users, RecMem showed no signs of quality collapse or "capacity saturation," maintaining stable performance throughout.
Limitations and Future Work
- ID-Based Focus: The current evaluation focuses on ID-based recommendations. The authors suggest that future work should explore richer textual or multimodal inputs.
- Fixed Allocation: The number of memory slots and segment lengths are currently fixed. Adaptive allocation—giving more memory to "complex" users—could be a promising direction.
- Theoretical Assumptions: The stability proof relies on specific assumptions like "contraction," which the authors approximate via spectral normalization but do not strictly certify.
Summary updated