CREAM: Continual Retrieval on Dynamic Streaming Corpora with Adaptive Soft Memory

HuiJeong Son, Hyeongu Kang, Sunho Kim, Subeen Ho, SeongKu Kang, Dongha Lee, Susik Yoon

TL;DR

CREAM (Continual REtrieval with Adaptive Soft Memory) is a new self-supervised framework that allows information retrieval systems to adapt to ever-changing data streams without needing human-labeled "ground truth." By using a dynamic "soft memory" to cluster evolving topics, it outperforms existing unsupervised methods by over 27% and even rivals supervised systems that have access to labeled data.

What Problem Does This Paper Address?

Most modern search engines and AI-powered retrieval systems (like those used in RAG) are trained on static snapshots of data. However, the real world is a "dynamic stream." Topics evolve, and the meaning of words shifts depending on the context. For example, a query about an "agent" might refer to a pharmaceutical drug in a medical context but a real estate professional in a business context.

When new data arrives, simply retraining a model from scratch is too expensive. On the other hand, "continual learning"—updating the model incrementally—often leads to catastrophic forgetting, where the model loses old knowledge while trying to learn new things. Existing solutions usually rely on "hard memory," which stores a fixed set of human-labeled query-document pairs. This is a bottleneck: in a fast-moving stream of web data, you don't have time for humans to label everything.

Methods

The authors introduce CREAM, which replaces static "hard memory" with a "Soft Memory" structure. It works through three main technical pillars:

  1. Fine-grained Similarity Estimation: Instead of squashing a whole document into a single vector (which loses detail), CREAM uses token-level interactions (inspired by ColBERT). This allows the system to detect subtle semantic shifts in how specific words are being used in new topics.
  2. Regularized Cluster Prototyping: To keep things efficient, CREAM groups similar documents and queries into clusters. Because documents have different lengths, they use Random Projection Locality Sensitive Hashing (RP-LSH) to create fixed-size "prototypes" (centroids) for these clusters. This reduces the computational math required by over 1,000x compared to raw token comparisons.
  3. Stratified Coreset Sampling: To train the model without labels, CREAM selects a diverse "coreset" of representative queries and documents from its soft memory. It then uses self-supervised contrastive learning, teaching the model that a query should be "close" to its most similar document in the cluster and "far" from documents in other clusters.

Datasets

The researchers evaluated CREAM on two major benchmarks, modified to simulate a 10-session "streaming" environment where topics appear, overlap, and disappear:

  • LoTTE: Covers five distinct domains (Writing, Recreation, Science, Technology, and Lifestyle) from StackExchange.
  • MSMARCO: A massive dataset from Bing queries, divided into five domains (e.g., Dated Events, Medical Treatments, Biology/Physics).

Key Insights and Ideas

  • Labels Aren't Always Better: The most surprising finding is that CREAM (unsupervised) often performs better than supervised methods. Why? Because supervised methods are anchored to a fixed set of "old" queries. CREAM’s soft memory is "topic-aware" and adapts its internal structure to the current distribution of data, making its training samples more relevant to the present.
  • The Power of "Soft" Structures: Unlike "hard" memory which stores specific pairs, "soft" memory stores the relationships between topics. This allows the system to "forget" outdated information gracefully by decaying old clusters while making room for emerging ones.
  • Efficiency via Hashing: By using RP-LSH to regularize prototypes, the authors proved that you can maintain high-fidelity semantic search without the massive "token-pairwise" computational tax usually associated with late-interaction models.

Key Results

CREAM significantly raised the bar for label-free continual retrieval:

  • Accuracy Boost: On average, it outperformed the strongest unsupervised baseline by 27.79% in Success@5 and 44.5% in Recall@10.
  • Rivaling Supervised Models: In many tests on the LoTTE dataset, CREAM achieved higher accuracy than state-of-the-art models that were actually given the correct answers (labels) during training.
  • Robustness: Ablation studies showed that the "fine-grained similarity" was the most critical component; removing it caused performance to drop by nearly 45%.

Limitations and Future Work

While CREAM is highly effective, it isn't "free." The authors noted that Query Selection—the process of picking the best training samples from memory—is computationally expensive, accounting for nearly 50% of the total processing time.

Future work aims to:

  • Scale to broader tasks: Moving beyond simple question-answering to tasks like real-time summarization.
  • Hierarchical Memory: Developing a multi-level soft memory to handle even more complex, nested topic shifts.
  • Agentic Integration: Turning the soft memory into a long-term "knowledge base" for AI agents that need to remember facts across long periods of time.

Summary updated

Abstract

Information retrieval (IR) in dynamic data streams is a crucial task, as shifts in data distribution degrade the performance of AI-powered IR systems. To mitigate this issue, memory-based continual learning has been widely adopted for IR. However, existing methods rely on a fixed set of queries with ground-truth documents, which limits generalization to unseen data, making them impractical for real-world applications. To enable more effective learning with unseen topics of a new corpus without ground-truth labels, we propose CREAM, a self-supervised framework for memory-based continual retrieval. CREAM captures the evolving semantics of streaming queries and documents into dynamically structured soft memory and leverages it to adapt to both seen and unseen topics in an unsupervised setting. We realize this through three key techniques: fine-grained similarity estimation, regularized cluster prototyping, and stratified coreset sampling. Experiments on two benchmark datasets demonstrate that CREAM exhibits superior adaptability and retrieval accuracy, outperforming the strongest method in a label-free setting by 27.79% in Success@5 and 44.5% in Recall@10 on average, and achieving performance comparable to or even exceeding that of supervised methods.
BibTeX
@inproceedings{son2026cream,
  author = {HuiJeong Son and Hyeongu Kang and Sunho Kim and Subeen Ho and SeongKu Kang and Dongha Lee and Susik Yoon},
  title = {{CREAM: Continual Retrieval on Dynamic Streaming Corpora with Adaptive Soft Memory}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2026},
  doi = {10.1145/3770854.3780281},
}
Figure