Imagine you want to create a smooth animation morphing a picture of a cat into a dog using a generative model. The standard approach is linear interpolation: you take the latent vector for the cat ($z_0$) and the dog ($z_1$), and step along a straight line ($z_t = (1-t)z_0 + tz_1$) using equal mathematical increments of $t$ (such as $0.0, 0.2, 0.4, \dots$).
Mathematically, your traversal speed across the latent space is perfectly constant. But when you watch the generated sequence, the transition looks jarring. For the first few frames, almost nothing changes. Then, in a sudden burst over one or two frames, the cat’s face violently warps into a dog before stalling again at the destination.
This happens because mathematical uniformity in a model's parameter space does not align with human visual perception. To fix this, a growing body of research focuses on perceptually uniform interpolation—abandoning linear steps in favor of non-linear transition schedules that ensure visual change progresses at a constant speed to the human eye.
Why Linearity Fails: The Non-Uniform Latent Manifold
The core issue is that the Euclidean geometry of a trained latent space is fundamentally warped relative to human perception. As Schwettmann et al. observe, the relationship between latent vector displacement and perceived visual change is highly non-linear and inconsistent across different regions of a model's space. A step of size $\Delta z$ might cause an imperceptible shift in a dense, highly sensitive region of the manifold, but cause a massive, chaotic jump in another.
This lack of "constant velocity" means that standard optimization and traversal methods struggle. Wang et al. point out that because Euclidean distance does not scale linearly with perceptual change, it is incredibly difficult to determine appropriate step sizes when navigating or searching a generative model's manifold.
This perceptual non-uniformity is a universal challenge across digital media representations:
- Material Appearance: Serrano et al. demonstrated that linear interpolation within the principal component space of material attributes leads to sudden, unpredictable visual changes rather than smooth shifts in texture or gloss.
- Artistic Abstraction: When simplifying a complex scene into a sparse sketch, Vinker et al. noted that automated systems create jarring transitions if they fail to model the non-linear way humans perceive the loss of geometric detail.
If the space itself is non-uniform, taking uniform steps guarantees a non-uniform experience. To achieve a smooth transition, we must change how we sample the path.
The Solution: Non-Linear Transition Scheduling
To make a transition feel smooth, the rate of visual evolution must be constant. If a segment of the latent path contains a rapid conceptual shift, the interpolation schedule must slow down and sample more densely in that zone. If a segment changes slowly, the schedule should take larger, sparse steps.
Standard Linear Sampling (Uniform parameter steps -> Jarring visual jumps):
Latent Space: [ x ---- x ---- x ---- x ---- x ]
Visual Change: [ None | None | BURST! | None ]
Perceptually Uniform Sampling (Non-linear parameter steps -> Smooth visual flow):
Latent Space: [ x ------- x --- x -- x ---- x ]
Visual Change: [ Smooth | Smooth | Smooth | Smooth ]
Recent papers have proposed two main paradigms to achieve this: reactive rescheduling (measuring and adjusting during generation) and proactive regularizing (fixing the space itself).
1. Reactive Rescheduling via Perceptual Metrics
One way to enforce uniformity is to actively measure visual change between steps using a perceptual metric (like LPIPS) and adapt the step sizes dynamically.
Instead of guessing a fixed schedule, Yang et al. employ a binary search algorithm guided by a perceptual similarity metric. The algorithm dynamically selects the next transition point so that the visual distance between consecutive frames remains perfectly constant, preventing the animation from stalling or jumping.
Similarly, Zhang et al. use a specialized sampling schedule that redistributes interpolation steps based on perceptual distance metrics. By allocating more frames to the high-velocity "burst" zones in the middle of an image-morphing sequence, they ensure the transformation appears to progress at a constant speed to an observer.
2. Statistical Priors (The Beta Distribution)
Running a binary search or calculating perceptual distances for every frame can be computationally expensive. An alternative is to use a statistical proxy that approximates where the bursts occur.
In text-to-image diffusion models, the most rapid conceptual changes typically occur in the middle of the interpolation path, where the model transitions from the source prompt's influence to the target's. To exploit this, He et al. utilize a non-linear Beta distribution to sample interpolation coefficients. Because a Beta distribution can be shaped to concentrate samples near the center of the $[0, 1]$ interval, it naturally allocates more computational steps to critical transition zones, smoothing out the perceived evolution without requiring an active search.
3. Proactive Regularization: Flattening the Manifold
Rather than fixing the interpolation schedule after a model is trained, can we force the latent space to be perceptually uniform by design?
Chen et al. introduce the Flat Manifold VAE (FMVAE), which addresses the root of the problem during the training phase. By introducing a novel regularization technique that penalizes variations in velocity across the latent space, they explicitly "flatten" the manifold. In an FMVAE, the latent space is successfully coerced into a consistent coordinate system where simple Euclidean distances accurately and linearly reflect true data similarity.
Conclusion
Whether by flattening the latent manifold during training (Chen et al.), dynamically searching for constant-distance steps (Yang et al.; Zhang et al.), or using non-linear sampling priors (He et al.), the core insight remains the same: smoothness is a property of human perception, not mathematical linearity. By shifting from uniform mathematical steps to uniform perceptual steps, these techniques allow generative models to create transitions that feel natural, predictable, and visually coherent.
Key Foundational Work
An intuitive control space for material appearance (2016) — contributed 2 concept elements to this direction.
Learning Flat Latent Manifolds with VAEs (2020) — contributed 1 concept elements to this direction.