Video generators, the diffusion models that turn a text prompt into a clip by gradually denoising random noise, often "cheat" when asked for change. Ask for a seed germinating or ice melting and they return a beautiful still image with a little camera wobble. The field calls this temporal stagnation. The usual metrics measure how much pixels move, so a model can score well by jittering or flickering, and poorly by producing a slow but real transformation. We propose to measure something else: whether the topology of the scene changes the way the prompt demands. Topology here means counts such as how many separate objects there are, how many holes they have, and which object split from or merged into which. We then use that measurement as a reward for post-training. A melting ice cube, a seed splitting, or a balloon bursting each leaves a short, discrete record of such events. We call this record an event trace. A frozen video leaves an empty trace. A video with objects "appearing out of thin air" leaves a trace full of events that have no parent object.
The design principle comes from a surprising place. In string-theory gauge theories built from branes at toric singularities, many different-looking graph descriptions are equivalent because they are related by local rewrite moves. The useful objects there are quantities that survive every such local move and change only when the underlying physical class changes. We borrow that lesson rather than the mathematics itself. Camera motion, texture flicker, and segmentation noise play the role of local moves, and the event trace is built to be invariant to them. Genuine physical transformations play the role of a change of class. The claim is that a motion signal built this way is harder to game than pixel-motion rewards and more faithful to what humans mean by "something happened," especially on long, metamorphic processes where current methods fail.
Core Claim
Temporal stagnation and the motion–artifact trade-off are two symptoms of one measurement problem. Current signals conflate nuisance change with physical change. Dynamic-degree-style scores reward any pixel displacement. Focal losses such as MotiF, which reweights the diffusion loss by optical-flow heatmaps, depend on a flow estimator that assumes pixels keep their identity. That assumption fails exactly when material melts, splits, or appears. Coherence scores such as the CHScore of ChronoMagic-Bench count lost tracking points, and so penalize the births and splits that metamorphosis requires. We claim that a discrete, nuisance-invariant event trace, compared against the trace of a real reference video, separates three cases that existing metrics blur together: frozen output, which has an empty trace; hallucinated change, which has events without lineage; and faithful change, whose trace matches the reference. We further claim that using this comparison as a segment-free, clip-level reward reduces stagnation without the reward hacking that motion-magnitude rewards invite.
Where the idea comes from, and what it does not borrow
Invariants of Toric Seiberg Duality shows that bipartite graphs on a torus, related by local "urban renewal" moves, share exact invariants. These include the complex structure of the torus under an embedding fixed by an extremization principle, its Klein $j$-invariant, and combinatorial quantities such as the kernel dimension of a generalized incidence matrix, which is fixed by global boundary data. Earlier work on toric duality as Seiberg duality established that these local moves are the physical equivalences. These papers supply the design principle: pick quantities that are provably blind to a named family of local rewrites and sensitive only to global changes. We do not claim that video frames are brane tilings, and we will not compute $j$-invariants of attention maps. The research file's Gap 1 suggests that direction, but no argument connects dimer invariants to motion quality, and we would expect it to fail.
The knots–quivers line (Knot-quiver correspondence: a brief review; Quivers for 3-manifolds) supplies the second principle. In that line, abstract integer invariants are earned by being counts of physical objects, namely BPS states or holomorphic disks. Our event trace is likewise a list of integer counts of physical things, objects and holes and their lineage, rather than a learned score.
A constraint comes from machine learning on quivers. Machines and Mathematical Mutations finds that a 4-layer graph network classifying mutation classes drops from 99.2% accuracy on 11-node quivers to 89.1% on 20-node quivers, because it cannot see cycles longer than its depth. Quiver mutations, Seiberg duality, and machine learning reports degradation on unseen classes. We read both as evidence that global invariants should be computed explicitly rather than learned by a local critic, so our trace is computed algorithmically, not predicted by a network.
Research Program
The first component is the trace itself. For each frame $t$, an off-the-shelf video segmentation model produces a region map, which we convert to a planar region-adjacency graph $G_t$. From $G_t$ we read the number of foreground components $b_0(t)$ and the number of enclosed holes $b_1(t)$. We link components across frames by mask overlap, which yields split, merge, birth (a component with no parent), and death events. Nuisance invariance is enforced in two ways. First, any change that reverts within a window of $w$ frames is discarded as a local move; this filters flicker and segmentation noise. Second, components are compared after compensating for estimated global camera motion. The trace $E$ of a clip is the ordered list of surviving events, each tagged with its type and coarse time.
The second component is the comparison. For a generated clip with trace $E_g$ and its reference video with trace $E_r$, the metamorphic fidelity score is
$$S(E_g, E_r) = 1 - \frac{d_{\text{edit}}(E_g, E_r)}{\max(|E_g|, |E_r|, 1)},$$
where $d_{\text{edit}}$ is the edit distance between the two event sequences, with substitution cost depending on event type. Alongside $S$ we report two diagnostic rates. The stagnation rate is the fraction of prompts with empty $E_g$ but nonempty $E_r$. The orphan rate is the fraction of births with no plausible parent under the prompt; this captures "a building appearing out of thin air."
The third component uses $S$ as a reward. We follow the preference-optimization recipe of DenseDPO, which builds structurally aligned video pairs by guided sampling and labels them. The difference is that preferences come from comparing $S$ over whole clips rather than from vision-language-model judgments on 1-second segments. For prompts without reference videos, the target trace is instead extracted from text: an event template, for example "one component splits into two, then $b_0$ grows," parsed from the prompt by a language model and checked by hand on a subset.
Baselines
The naive reference is the untouched base model. We will use OpenSoraPlan, the model ChronoMagic-Bench fine-tunes, and AnimateDiff, which ByTheWay and IV-mixed Sampler both modify. On VBench Dynamic Degree, ByTheWay reports AnimateDiff at 0.40 vanilla and 0.52 with its Fourier attention scaling.
The strongest competitor for the training question is DenseDPO. It targets exactly the failure we target, preference tuning that rewards stillness, and it reports a Dynamic Degree of 85.38 on VideoJAM-bench against 80.25 for vanilla DPO, using 10k rather than 30k labeled pairs. We will reimplement it on the same base model, with the same 10k pairs budget, the same LoRA rank, and the same early-stopping rule. The only change is the labeling source: vision-language-model segment labels versus trace-derived clip labels. Beating it means higher $S$ and lower stagnation rate on ChronoMagic-Bench at a Dynamic Degree and VBench quality score no worse than DenseDPO's, and a human-preference win on metamorphic prompts.
The strongest data-centric competitor is fine-tuning on ChronoMagic-Pro time-lapse data, as in ChronoMagic-Bench and MagicTime. MagicTime reports FVD 441.17 on its ChronoMagic set. IV-mixed Sampler reports AnimateDiff UMT-FVD on ChronoMagic-Bench falling from 275.2 to 228.6. We will match the 10,000-sample fine-tuning budget and test whether trace-based preference tuning adds to data curation or merely duplicates it.
For the metric question, the competitors are Dynamic Degree, MTScore, and CHScore, plus a whole-clip vision-language judge. The ablations isolate the key ingredient. One replaces the trace reward with mean optical-flow magnitude, which is MotiF's signal used as a reward. Another removes the local-move filter, using raw per-frame component counts. We will also report a reward that counts events regardless of whether they match the reference. If the unfiltered or reference-free versions do as well, then nuisance invariance and reference matching are not what matters.
Why not just…?
Why not just amplify temporal attention at sampling time, as ByTheWay does? It is cheap and it works on motion magnitude. However, its authors report that raising the motion factor produces tearing and distortion, and that the right settings differ across base models. It scales how much things move, not whether the right events happen. We will run it at its reported settings and measure its orphan rate. Our prediction is that its Dynamic Degree gains come with more orphan births and no gain in $S$ on metamorphic prompts.
Why not just reweight the loss by optical flow, as MotiF does? MotiF reports a 72% average human preference over nine baselines on TI2V-Bench, but it trains on a million pairs and its flow estimator assumes pixels persist. On melting, splitting, and smoke, flow is undefined or wrong. The flow-magnitude reward ablation tests this directly. If it matches the trace reward on ChronoMagic-Bench, our claim fails.
Why not just ask a vision-language model to judge the whole clip? This is the real threat. DenseDPO's 1-second limit is a design choice, not a law, and a strong judge over a full clip might recognize "the seed split" as well as our trace does. We take it seriously in three ways. We include a whole-clip judge as both a metric baseline and a reward baseline under the same pair budget. We measure agreement of each signal with human event annotations. We test robustness by adding camera shake and flicker to otherwise frozen clips: a judge that rates shaken stills as dynamic is hackable in the way we hypothesize. If the judge matches the trace on agreement and robustness, the honest conclusion is that the trace's value is its cost and transparency, not its accuracy.
Why not just curate more time-lapse data? ChronoMagic-Bench finds that even strong closed systems fail long-horizon metamorphosis after exposure to dynamic data. We still treat curation as complementary and test whether trace-based tuning helps on top of it.
Proposed Experiments
The first experiment validates the trace before any training. We build a controlled set by taking ChronoMagic-Bench reference videos, which carry real events, and producing three variants of each: frozen copies of the first frame; copies with synthetic camera shake, flicker, and texture jitter; and copies with spliced-in objects that have no lineage. A valid metric must give the real videos high $S$ against themselves. It must give shaken stills the same score as frozen stills, which is invariance to local moves, and it must flag splices as orphans. Dynamic Degree, CHScore, MTScore, and the whole-clip judge are scored on the same variants. The decisive check is whether the trace is the only signal that ranks real above shaken above frozen while flagging splices. We will also collect human event annotations on a few hundred ChronoMagic-Bench clips across its Biological, Human Creation, Meteorological, and Physical categories, and report rank correlation of each signal with human judgments of "the right thing happened." For comparison, VBench reports Spearman correlations often above 0.9 for its own dimensions, but no number on Lacuna exists for metamorphic event correctness.
The second experiment is the training comparison described in the Baselines section. It runs on OpenSoraPlan and AnimateDiff and is evaluated on ChronoMagic-Bench (1,649 prompts with references), TI2V-Bench (320 prompts), VBench, and VideoJAM-bench. Metrics are $S$, stagnation rate, orphan rate, Dynamic Degree, VBench quality dimensions, and blinded human preference. The decisive outcome is a drop in stagnation rate without a rise in orphan rate relative to DenseDPO at equal label budget. A drop in stagnation paired with a rise in orphans would mean we had only moved the trade-off.
The last check guards against the reward being gamed through the segmenter. We will retrain the reward with one segmentation model and evaluate $S$ with a different one. Gains that vanish under the held-out segmenter would show that the policy learned to fool the segmenter rather than to animate the scene.
References
- MotiF: Making Text Count in Image Animation with Motion Focal Loss
- ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation
- INVARIANTS OF TORIC SEIBERG DUALITY
- Toric duality as Seiberg duality and brane diamonds
- Knot-quiver correspondence: a brief review
- Quivers for 3-manifolds: the correspondence, BPS states, and 3d $$ \mathcal{N} $$ = 2 theories
- Machines and Mathematical Mutations: Using GNNs to Characterize Quiver Mutation Classes
- Quiver mutations, Seiberg duality, and machine learning
- DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models
- ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way
- IV-mixed Sampler: Leveraging Image Diffusion Models for Enhanced Video Synthesis
- MagicTime: Time-lapse video generation models as metamorphic simulators