Measuring Only What Motion Ignores: Weak, Motion-Blind Identity Guidance for Video Diffusion

Video diffusion models are generators that turn random noise into a clip by denoising it step by step. They face a well-documented trade-off. If you push frames to look like a reference image of the subject, the video freezes. If you let frames move freely, the subject's face, clothing or fur slowly morphs. Today's fixes share one feature: they make each frame copy features from a reference frame. One example shares attention features across frames (RefDrop). Another injects the attention "query" features (Motion by Queries). A third inserts a full anchor frame (Anchor Frame Bridging). The copied features also carry where things are, so the more identity you enforce, the more motion you erase.

Quantum measurement theory has studied the same tension for decades: reading out a system's state disturbs it. It offers three lessons that transfer directly. First, observe weakly rather than sharply. Second, observe only quantities that are insensitive to the dynamics you want to keep. Third, when many copies of a system are available, observe their average, so that each copy is barely disturbed. We propose identity guidance that applies all three. At each denoising step it nudges only position-invariant statistics of the subject's features, meaning the channel means and variances over the subject region. It nudges them weakly, it pools them over all frames as an ensemble, and it sets the nudge strength by a feedback rule driven by a running estimate of how far identity has drifted. The claim is that this guidance changes motion very little to first order, and that it should give a better identity-versus-motion trade-off curve than reference-feature copying, at no training cost.

Core Claim

The trade-off is not fundamental at the level of the guidance signal. It arises because current methods "measure" features that do not commute with motion. A reference feature at a given token encodes both appearance and location. Interpolating toward it, which RefDrop proved is what cross-frame attention sharing amounts to, shrinks variance between frames along every direction, including motion. Motion by Queries makes the problem explicit: query features encode both "what" and "where," so injecting them locks layout and leaks identity between same-class subjects.

By contrast, the mean and variance of feature channels pooled over a subject's region are unchanged when the subject moves rigidly. This holds because pooling discards location, so permuting spatial positions leaves these statistics the same. Correcting these statistics therefore changes appearance while leaving layout and optical flow untouched to first order. In quantum terms, this is the idea of a measurement whose observable commutes with the quantity being protected.

The ensemble lesson comes from Quantum feedback with weak measurements. That paper shows that weakly measuring the average over $N$ identical systems, then feeding the estimate back, steers the ensemble while the disturbance to any single member shrinks as $N$ grows. Frames of a clip are our ensemble. Identity is a property shared by all of them, and motion is how each one deviates.

The "weak and adaptive" lesson comes from Time-series quantum reservoir computing with weak and projective measurements. There, sharp readout (coupling $g=10$) wiped out the reservoir's memory of past inputs. Weak readout ($g\approx0.3$) preserved that memory and approached the unperturbed correlation capacity of $C=0.98$. We treat these quantum papers as design guidance, not as physics that literally governs diffusion latents. The video claims stand or fall on the video experiments.

Mechanism

At denoising step $t$, for each frame $f$, we take features $h_{f}$ from selected attention layers of the model's prediction. We obtain a soft subject mask from the cross-attention map of the subject's text token. We then compute masked channel statistics $s_f = (\mu_f, \sigma_f)$.

The identity error is $e_t = \bar{s}_t - s_{\text{ref}}$. Here $\bar{s}_t$ is the average of $s_f$ over all frames, and $s_{\text{ref}}$ is the statistic of the reference image. In text-to-video settings without a reference image, $s_{\text{ref}}$ is the statistic of the first frame, frozen once its estimate stabilises.

Early in denoising, the model's estimate of the clean video is noisy. We therefore keep a filtered estimate $\hat{e}_t$, an exponential moving average whose weight grows as the noise level falls, analogous to the quantum filter in Dong et al.. Each frame's features are then shifted by a common amount:

$$h_f \leftarrow h_f - g_t\,\Pi(\hat{e}_t), \qquad g_t = \min\!\left(g_{\max},\; k\,\frac{\|\hat{e}_t\|}{\|\hat{e}_t\| + \beta\,\sigma_t^2}\right)$$

In this rule, $\Pi$ maps a statistic correction back onto features within the mask, as a renormalisation of channel mean and scale. The noise level at step $t$ is $\sigma_t$. The constants $k$ and $g_{\max}$ set the gain and its ceiling, and $\beta$ discounts corrections when the estimate is unreliable.

Two properties make this rule distinct from existing methods. First, every frame receives the same correction, computed from the ensemble mean, so differences between frames are preserved. Second, the correction is large only when drift is both present and reliably estimated, so guidance is not a constant pull.

We also measure the disturbance, or back-action, directly. For a fixed seed, we compare optical-flow magnitude and inter-frame latent variance with and without guidance. The expectation is that this cost stays near zero until $g_t$ grows large enough for higher-order effects to matter.

Research Program

The work has three parts. The first verifies the motion-blindness property on its own. We invert real DAVIS clips taken from the FLF2V set (Anchor Frame Bridging), apply each guidance method at matched identity gain, and measure the change in RAFT optical flow against the unguided reconstruction. If statistic coupling does not disturb flow far less than RefDrop-style interpolation at equal identity gain, the central premise fails, and we would report that.

The second part traces full trade-off curves on the base models the cited baselines actually used. These are Stable Video Diffusion, which RefDrop used; VideoCrafter2, which Motion by Queries compared against; and Wan2.1-I2V, which Anchor Frame Bridging used. Every method in this comparison has a strength knob, so we compare curves rather than single settings.

The third part probes the known failure modes. These are same-class leakage, such as two horses or two people in one scene, as reported in Motion by Queries; fast motion; and fine positional detail such as text or logos on clothing. We expect our method to be weakest on fine positional detail, because position-invariant statistics cannot encode where a logo sits on a shirt. This limitation is foreseeable. We will quantify it on subject crops with DreamSim rather than hide it. A follow-up would apply the same rule to statistics in coordinates relative to the subject.

Baselines

The naive reference point is the unguided base model, sampled with the same seeds and prompts. Its subject consistency and dynamic degree anchor both ends of every curve.

The strongest training-free competitor is RefDrop. It concatenates reference keys and values into each frame's attention, with a global scalar $c$ controlling strength. It is the strongest baseline because it is the cleanest existing knob for exactly this trade-off, and it works on both UNet and transformer backbones. On its 220-prompt video set with Stable Video Diffusion, it reports a warping error of 0.0058 versus 0.0077 for vanilla SVD.

Warping error rewards frozen video, so beating RefDrop cannot mean lowering warping error alone. It means a curve of VBench subject consistency (DINO similarity) against VBench dynamic degree (RAFT flow) (VBench) that lies above RefDrop's across the range of motion. A matched comparison would be higher DINO similarity at equal dynamic degree, with warping error no worse than 0.0058 at the operating point that matches RefDrop's identity score.

We will sweep $c$ over the same number of settings as our $(k, g_{\max})$ grid. We will also give RefDrop a per-step schedule tuned with the same search budget, so that it is not handicapped by its global scalar.

The second strong competitor is Motion by Queries. It injects query features during the first 40% of denoising and adds framewise shared attention. It reports a multi-shot consistency score of 68.8 versus 63.2 for VideoCrafter2. We run its released schedule on VideoCrafter2 using its own evaluation suite. Beating it means exceeding 68.8 at equal or higher dynamic degree, and reducing same-class leakage in a two-subject test set.

For long multi-shot narratives we report Corgi on MainCharacter21. Corgi reports a 59.2% gain in long-term identity consistency and a 7.6% gain in diversity over FreeNoise and Gen-L-Video. Because Corgi fine-tunes per subject, we report its numbers together with its compute cost rather than claiming parity. Our target is to match its consistency without fine-tuning.

For image-to-video, TRIP reports F-Consistency₄ of 95.36% and FVD of 38.9 on WebVid-10M. This is a trained ceiling. We treat it as a reference point, not as an equal-compute competitor.

Three ablations isolate the ingredients. The first is constant-strength statistic matching, an AdaIN-style variant with no filter and no feedback. It tests whether the weak, adaptive coupling adds anything beyond choosing the right observable. The second uses the same filter and feedback but couples position-dependent features, as RefDrop does. It tests whether the choice of observable is what protects motion. The third matches each frame individually to the reference instead of matching the ensemble mean. It tests the ensemble argument.

Why not just…?

Why not just tune RefDrop's slider, or give it a per-step schedule? This is the most direct threat, and the baselines are built to settle it. RefDrop's own analysis shows that its operation interpolates toward reference features. Those features encode position, so interpolating toward them shrinks variance between frames along every direction at any $c$. A schedule changes when that shrinkage happens, not what it removes. If a tuned schedule reaches the same frontier as our method, the observable-choice claim is wrong. The second ablation, position-dependent coupling with our own controller, separates schedule effects from observable effects.

Why not just turn guidance on when a DINO identity score drops? That is a feedback controller, and it is close to half of our method. The difference is where the corrective push lands. A threshold controller that switches on RefDrop or query injection still pushes through features that carry motion, so every correction costs motion. Our feedback acts on statistics that ignore position. The second ablation is exactly this alternative, and our curve should dominate it if the observable matters.

Why not just match feature statistics AdaIN-style and drop the measurement framing? This is a real threat. If constant-strength statistic matching performs as well as the full method, then the quantum lens contributed only the idea of choosing a motion-blind observable. We would report it that way. Our prediction that it falls short rests on the reservoir result in Mujal et al., where coupling strength had a sharp optimum, and on our expectation that strong early-step corrections act on noisy predictions of the clean video. The first ablation tests this prediction directly at each step.

Why not train a decoupled identity/motion model like TRIP, or apply orthogonal projection as in Spectral Collapse in Diffusion Inversion? TRIP needs training on WebVid-10M and is tied to one architecture, whereas our method is applied at inference on any backbone. Orthogonal Variance Guidance projects corrections away from a structural gradient in static images. A temporal version would require computing a motion gradient at every step. Our approach avoids that cost, because position-invariant statistics are insensitive to rigid motion by construction. A temporal orthogonal-projection variant is a reasonable extra baseline if compute allows. We would not dismiss it.

Decisive Checks

Three results would settle the proposal. First, on inverted real clips at matched identity gain, statistic coupling should change RAFT flow substantially less than RefDrop does. Second, on SVD, VideoCrafter2 and Wan2.1-I2V, the curve of subject consistency against dynamic degree should lie above that of the schedule-tuned RefDrop and the threshold controller. Third, the method should beat Motion by Queries' 68.8 multi-shot score without losing dynamic degree, and should show lower leakage between same-class subjects.

Failure of the first check refutes the mechanism. Failure of the second while the first holds would mean that motion-blindness at the feature level does not survive the model's nonlinear decoding into pixels. That would itself be a useful negative finding about where entanglement between identity and motion arises inside these models.