MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narratives

Sihui Ji, Xi Chen, Shuai Yang, Xin Tao, Pengfei Wan, Hengshuang ZhaoarXiv 2025 · DOI · arXiv

Abstract

Generating long-form videos with high narrative consistency remains a significant challenge in AI research. This paper presents MemFlow, a novel framework that utilizes a dynamic, "flowing" memory mechanism for streaming text-to-video generation. By leveraging semantic retrieval guided by text prompts and employing sparse activation, MemFlow maintains visual coherence over extended durations (60+ seconds) while achieving a near-real-time generation speed of 18.7 FPS.

Introduction

Current text-to-video (T2V) systems often struggle with the "memory-efficiency trade-off." When generating long videos in chunks, models must remember previous context to ensure characters and settings do not "drift." Traditional approaches either use a fixed window of recent frames—which risks forgetting early narrative elements—or compress past frames into a small set of tokens, leading to a loss of fine-grained detail. MemFlow addresses these limitations by treating memory as an adaptive bank that retrieves relevant historical context based on the evolving story.

Methodology

The MemFlow architecture is built upon two core innovations designed to balance narrative depth with computational efficiency.

Narrative Adaptive Memory (NAM)

NAM replaces static memory windows with a Semantic Retrieval mechanism. Instead of simply looking at the most recent frames, the model uses the text prompt for the current video chunk to query a memory bank containing visual Key-Value (KV) caches of all previously generated frames.

Clip 1: A child plays with a ball on a beach, establishing the initial narrative context at the 12-second mark.

By calculating attention scores between text tokens and historical visual tokens, NAM pulls the most semantically relevant context. To maintain efficiency, MemFlow employs a "Redundant Removal" heuristic, condensing 5-second chunks into single "prototype" frames, which captures the essential visual information without overwhelming the system.

Sparse Memory Activation (SMA)

To prevent the large memory bank from slowing down the attention mechanism, MemFlow utilizes Sparse Memory Activation. SMA strategically activates only the top-k most relevant tokens in the memory bank for each query. This approach significantly reduces computational overhead, allowing the model to maintain high performance even as the video length increases.

Experimental Results

MemFlow was evaluated using the Wan2.1-T2V-1.3B architecture on the LongLive Switch Prompt Dataset and the VBench-Long benchmark.

Clip 4: The narrative progresses to the child hugging a dog at the 48-second mark, showcasing MemFlow's ability to maintain character consistency across clips.

Performance and Efficiency

  • Speed: MemFlow operates at 18.7 FPS on a single NVIDIA H100, representing a 38x speedup over competitors like SkyReels-V2.
  • Visual Quality: On VBench-Long, MemFlow achieved a Quality Score of 85.02 and an Aesthetic Score of 61.07, outperforming other long-form models such as LongLive and FramePack.
  • Consistency: Qualitative analysis shows that MemFlow effectively prevents "character drifting" and the generation of redundant subjects, ensuring a smooth narrative flow even as the prompt changes from playing with a ball to interacting with a dog.

Conclusion

MemFlow demonstrates that efficient long-form video generation does not require massive memory compression. Instead, by using text-guided sparse retrieval, models can maintain high-fidelity narrative threads across long durations. Future work will explore more sophisticated representative frame selection and multi-character interactions to further push the boundaries of AI-generated cinematography.

Summary updated

Abstract

The core challenge for streaming video generation is maintaining the content consistency in long context, which poses high requirement for the memory design. Most existing solutions maintain the memory by compressing historical frames with predefined strategies. However, different to-generate video chunks should refer to different historical cues, which is hard to satisfy with fixed strategies. In this work, we propose MemFlow to address this problem. Specifically, before generating the coming chunk, we dynamically update the memory bank by retrieving the most relevant historical frames with the text prompt of this chunk. This design enables narrative coherence even if new event happens or scenario switches in future frames. In addition, during generation, we only activate the most relevant tokens in the memory bank for each query in the attention layers, which effectively guarantees the generation efficiency. In this way, MemFlow achieves outstanding long-context consistency with negligible computation burden (7.9% speed reduction compared with the memory-free baseline) and keeps the compatibility with any streaming video generation model with KV cache.
BibTeX
@misc{ji2025memflow,
  author = {Sihui Ji and Xi Chen and Shuai Yang and Xin Tao and Pengfei Wan and Hengshuang Zhao},
  title = {{MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narratives}},
  year = {2025},
  eprint = {2512.14699},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.14699},
}
Figure