Eliminating Temporal Flickering in Generative Video Synthesis

62 concepts61 papers1 proposal

When state-of-the-art 2D generative models—such as StyleGAN or Diffusion Models—are applied to video sequences frame-by-frame, the results are often visually jarring. Even if every individual frame looks photorealistic, playing them in sequence reveals a chaotic dance of shifting textures, morphing backgrounds, and shimmering details.

This phenomenon is known as temporal flickering or inter-frame inconsistency. Resolving it is the central hurdle in transitioning generative AI from static image synthesis to fluid, believable video editing and generation.


The Root Cause: Stochastic Independence

To understand why temporal flickering happens, we have to look at how generative models create details.

When a generative model synthesizes an image, it relies on stochastic processes—such as sampling from a random noise distribution or optimizing an unconstrained latent code. When you process a video frame-by-frame, the model treats each frame as an isolated universe. It has no memory of what it did a fraction of a second ago.

As a result, minor variations between consecutive frames (like a slight shift in lighting or a camera nudge) cause the model to make completely different generative choices for the exact same semantic object. As noted by Ceylan et al., this inherent randomness means the model interprets text prompts or conditioning factors differently from frame to frame, causing colors and textures to shift unnaturally over time.

This lack of temporal grounding manifests in three distinct visual artifacts:

  1. Texture Crawling and Boiling: Fine-grained details like hair, teeth, or clothing patterns morph unpredictably between frames, creating a distracting "boiling" effect (Chen et al.).
  2. Texture Sticking: An opposite but equally unnatural artifact where fine textures appear fixed to specific pixel coordinates on the screen rather than moving naturally with the underlying object geometry (Karras et al.).
  3. Identity Drift and Background Shifting: Static elements like backgrounds or a person's facial structure warp or change appearance because the model lacks a mechanism to enforce global layout coherence across the temporal dimension (Shin et al.).

Core Strategies for Temporal Coherence

To eliminate flickering, researchers have developed several distinct paradigms that introduce cross-frame awareness into the generative pipeline.

1. Feature Propagation and Attention Alignment

Instead of letting each frame generate its internal representations from scratch, we can force the model's hidden layers to share information across time.

A prominent approach is to manipulate the internal attention maps of diffusion models. For instance, Geyer demonstrated that explicitly propagating edited internal features ("tokens") across frames based on the original video's motion eliminates texture crawling. Similarly, Ceylan et al. utilize self-attention feature injection from anchor frames to guide the generation of subsequent frames. By aligning temporal attention maps across different decoder blocks, the model maintains a stable global layout and avoids structural warping (Bu et al.).

2. Decoupling Spatial Appearance and Temporal Dynamics

Another powerful intuition is to separate the task of generating high-quality details from the task of ensuring smooth motion.

Wang showed that video generation models can be scaled effectively by decoupling spatial appearance (learned from static images) from temporal dynamics (learned from unlabeled video clips). This concept is highly effective in video super-resolution and editing. For example, Sun et al. decoupled spatial detail generation and temporal consistency into two specialized Low-Rank Adaptation (LoRA) modules, allowing the model to hallucinate sharp details without letting them flicker. Shi 0001 et al. similarly bridged task-specific image models with video diffusion models to separate content creation from temporal smoothing.

3. Geometric and Flow-Guided Anchoring

If we can track how pixels move in the real world, we can use that geometry to anchor the generative process.

Early video inpainting frameworks, such as VORNet (Chang et al.), combined optical flow warping with recurrent refinement networks to reuse historical background data rather than generating new pixels for every frame. For highly dynamic subjects like human faces, incorporating 3D Morphable Models (3DMM) or 3D-aware landmarks allows models to anchor details like teeth and eyes to a stable structural skeleton, preventing the stochastic hallucinations that cause facial jitter (Ostrek et al.; Zhao et al.; Zhang et al.).

4. Spatiotemporal Slice Inversion

A creative alternative to sequential frame processing involves changing how the generative model views the video volume entirely.

Instead of feeding a model standard spatial frames ($XY$) over time ($T$), Cohen et al. proposed slicing the video volume along the spatiotemporal planes ($XT$ or $YT$). By treating these spatiotemporal slices as "images" and denoising them with a standard text-to-image diffusion model, the continuous motion and structural integrity of the video are naturally preserved. Because time is baked directly into the slices, the frame-by-frame consistency trap is bypassed entirely.

Standard Processing:  [Frame 1 (XY)] -> [Frame 2 (XY)] -> [Frame 3 (XY)]  --> Independent & Flickering
Slicedit Paradigm:    [Slice 1 (XT)] -> [Slice 2 (XT)] -> [Slice 3 (XT)]  --> Time-consistent by Design

Conclusion

The shift from per-frame image editing to true video synthesis relies on moving away from independent generation. Whether by locking internal attention maps across time, decoupling appearance from motion, or leveraging underlying 3D geometry, the goal remains the same: ensuring that the model's generative choices are grounded in what came before and what follows.

Key Foundational Work

VORNet: Spatio-Temporally Consistent Video Inpainting for Object Removal (2019) — contributed 2 concept elements to this direction.
Video Inpainting by Jointly Learning Temporal Structure and Spatial Details (2019) — contributed 1 concept elements to this direction.
Learning temporal coherence via self-supervision for GAN-based video generation (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_flickering_inter_frame_inconsis_98a180df,
  title = {temporal flickering (inter-frame inconsistency)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-flickering-inter-frame-inconsistency/rdr_98a180df22de49e0af2b14c242c8e271},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_temporal_flickering_inter_frame_inconsis_98a180df_v1,
  title = {temporal flickering (inter-frame inconsistency)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-flickering-inter-frame-inconsistency/rdr_98a180df22de49e0af2b14c242c8e271/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake