EgoLCD: Egocentric Video Generation with Long Context Diffusion

Liuzhou Zhang, Jiarui Ye, Yuanlei Wang, Ming Zhong, Mingju Cao, Wanke Xia, Bowen Zeng, Zeyu Zhang, Hao TangarXiv 2025 · DOI · arXiv

Introduction

Generating coherent, long-form video from a first-person (egocentric) perspective remains a significant challenge in computer vision. While current diffusion models excel at short clips, they often suffer from "content drift"—a phenomenon where the visual identity of objects and environments degrades or transforms over time. EgoLCD addresses this by reframing video synthesis as a memory management problem, utilizing a dual-memory architecture to ensure long-term consistency.

The Challenge of Content Drift

In egocentric video, the camera is typically mounted on a user's head, leading to rapid motion and frequent occlusions. Standard autoregressive models predict the next frame based on a limited window of previous frames. As the sequence length increases, the model's "memory" becomes cluttered or fades, leading to "generative forgetting."

Egocentric video generation involves complex interactions and dynamic viewpoints, as seen in this first-person scene.

A kitchen that starts with blue tiles might gradually morph into a different room, or a specific tool might change shape mid-action. Standard metrics often miss this because they average quality over the whole video, ignoring the fact that the end of the video no longer matches the beginning.

Methodology: The Dual-Memory Architecture

The EgoLCD (Egocentric Long Context Diffusion) framework introduces a "dual-memory" architecture designed to keep the video on track over extended durations. The core insight is that long-form video is not just about predicting the next pixel, but about managing what information to keep and what to discard.

  1. Long-Term Sparse KV Cache: This serves as the model's "global anchor." It stores historical key-value pairs from previous segments. To prevent computational costs from exploding, it uses a sparsity mechanism that only retains the most "important" tokens based on attention scores. This allows the model to maintain global coherence across long sequences.
  2. Short-Term Memory with LoRA: Immediate context is handled via a standard attention window enhanced with Low-Rank Adaptation (LoRA). These parameters act as implicit memory units, helping the model quickly adapt to local changes—such as a sudden head turn—without disrupting the underlying scene structure.
  3. Memory Regulation Loss: This is a crucial training innovation. It forces the model to align its current generation with semantically retrieved "anchors" from its long-term memory. By "tethering" the diffusion process to historical data, the model is prevented from hallucinating new, inconsistent environments.

To guide the generation process, EgoLCD also uses Structured Narrative Prompting (SNP). By leveraging large language models like GPT-4o, the system generates a temporally ordered script of detailed captions, ensuring that actions follow a logical, human-centric sequence.

Evaluation and Results

The researchers evaluated EgoLCD on the EgoVid-5M dataset, a massive-scale benchmark for egocentric activities. To better quantify long-term stability, they introduced the Normalized Referenced Drifting Penalty (NRDP), which specifically measures how much a video's quality and consistency deviate from the initial frame over time.

EgoLCD maintains visual consistency across hundreds of frames, preventing the 'content drift' common in long-form video generation.

EgoLCD achieved state-of-the-art results, particularly in consistency metrics. For instance, its Subject Drift score was 0.0844, significantly lower than competing models like MAGI (0.6618) and Framepack (4.3984). Ablation studies confirmed that the Memory Regulation Loss is critical; removing it led to a sharp increase in visual drift, proving that architectural memory requires explicit supervision to function correctly.

Conclusion

By treating long-form video generation as a memory management task rather than a simple sequence prediction, EgoLCD sets a new standard for egocentric synthesis. While challenges remain regarding GPU resource requirements and prompt sensitivity, the use of sparse long-term caches and explicit memory regulation provides a robust path forward for creating stable, minute-long first-person videos.

Summary updated

Abstract

Generating long, coherent egocentric videos is difficult, as hand-object interactions and procedural tasks require reliable long-term memory. Existing autoregressive models suffer from content drift, where object identity and scene semantics degrade over time. To address this challenge, we introduce EgoLCD, an end-to-end framework for egocentric long-context video generation that treats long video synthesis as a problem of efficient and stable memory management. EgoLCD combines a Long-Term Sparse KV Cache for stable global context with an attention-based short-term memory, extended by LoRA for local adaptation. A Memory Regulation Loss enforces consistent memory usage, and Structured Narrative Prompting provides explicit temporal guidance. Extensive experiments on the EgoVid-5M benchmark demonstrate that EgoLCD achieves state-of-the-art performance in both perceptual quality and temporal consistency, effectively mitigating generative forgetting and representing a significant step toward building scalable world models for embodied AI. Code: https://github.com/AIGeeksGroup/EgoLCD. Website: https://aigeeksgroup.github.io/EgoLCD.
BibTeX
@misc{zhang2025egolcd,
  author = {Liuzhou Zhang and Jiarui Ye and Yuanlei Wang and Ming Zhong and Mingju Cao and Wanke Xia and Bowen Zeng and Zeyu Zhang and Hao Tang},
  title = {{EgoLCD: Egocentric Video Generation with Long Context Diffusion}},
  year = {2025},
  eprint = {2512.04515},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.04515},
}
Figure