Do Fast Motion Generators Erase the Rare, Trembling Patients?

Diffusion and flow models are generators that turn random noise into a sample by taking many small denoising steps. They are now the default tool for synthesizing human motion, both speech-driven conversational gestures and, increasingly, "virtual patients" meant to fill the rare severe classes of clinical motor datasets. Because many sequential steps are slow, the field compresses them into a few steps or one. This note argues that the compression is not neutral. A few-step generator trained with a squared-error objective behaves like a conditional average. Averaging removes two things first. The first is high-frequency motion with random phase, such as tremor, freezing tremble, or finger flutter. The second is classes with few training examples, which get pulled toward the common classes. In Parkinsonian motion these two coincide: the scarce severe patients are the ones whose signature lives in fast, irregular movement. A fast generator can therefore report good average quality while producing "severe" patients who move like mild ones. A classifier that is then trained on those samples learns the wrong boundary.

We propose to measure this failure directly and to fix it with a generator that splits motion by frequency band rather than by body part. The slow, gross trajectory is generated in one or two straight-flow steps. The fast residual gets a small, separate, class-balanced generator with its own step budget. Total latency is held equal to existing few-step systems. The test is concrete. Synthetic severe-gait samples on CARE-PD, and synthetic rare-emotion gestures on a long-tailed version of BEAT, should keep their class-specific spectral content and be recognized as their class by a classifier trained only on real data. We then ask whether they improve rare-class recall under leave-one-site-out evaluation.

Core Claim

The open problem is not "apply a fast generator to clinical motion." That is a straightforward engineering step. The open question is whether step reduction costs the tail classes more than the head classes, and whether that cost is concentrated in high frequencies. The mechanism is simple. Consider a one-step map trained to regress data from noise under a squared loss. For an oscillation $x_H(t) = a \sin(\omega t + \phi)$ whose phase $\phi$ is unpredictable from the conditioning, the loss-minimizing output converges toward $\mathbb{E}[x_H] = 0$. The tremor averages away even when its amplitude $a$ is large. For a class with little data, the conditional average is further dragged toward neighboring common classes. This is the "feature borrowing" that CORAL documented in the bottleneck of image diffusion models trained on long-tailed data.

The file already notes the conversational side of this tension. In the GENEA Challenge 2022, synthetic gestures out-scored real motion capture on human-likeness (median 71 vs. 70, full-body tier), plausibly because they smooth away jitter. The GTSN tremor work shows that the clinically decisive signal is exactly such subtle, fast movement: it had to amplify it with Eulerian video magnification to score it. Our claim is that the tension becomes worse when steps are removed, and worst for the rarest classes.

The fix is to give the fast band its own generative budget and class-balanced training. The analogy is DIDiffGes, which gave fingers separate noise so torso motion would not swamp them. We split by frequency rather than by limb, because in clinical motion the pathological band cuts across limbs.

Relation to Prior Work

Three works supply the method. Match-TTSG generates speech and upper-body gesture jointly with optimal-transport conditional flow matching, which learns near-straight noise-to-data paths, in a 1D Transformer U-Net. It reports a 15-fold real-time-factor speedup over diffusion (0.13 vs. 1.94). We reuse this architecture as our many-step teacher and low-band generator. DIDiffGes reaches 10-step gesture sampling (4.4 s of motion in about 0.4 s on a V100) with a GAN discriminator inside semi-implicit denoising, plus decoupled body and hand noise. We borrow the decoupling idea but change its axis from body part to frequency band. Mean Flows learns average velocities for one-step generation. It is the most aggressive compression we test, and the one our averaging argument predicts will hurt tails most. All three were evaluated only on healthy, roughly balanced motion or images, and none reports per-class or per-frequency fidelity.

Two works motivate the proposal by exposing the gap. CARE-PD pools 8,477 Parkinsonian walking segments from 8 clinical centers as SMPL meshes. It shows that healthy-motion pretraining transfers poorly (MPJPE 60.7 mm vs. 7.5 mm after pathological pretraining) and names the scarcity of UPDRS-gait score 3 as a bottleneck. It proposes synthetic data as future work but builds no generator. GTSN reports that MDS-UPDRS scores 3–4 are too scarce for reliable multi-class accuracy even with mirroring augmentation.

One work constrains the design and supplies a component. CORAL adds a supervised contrastive loss on the U-Net bottleneck through a projection head that is discarded after training. It improves ImageNet-LT FID from 17.08 to 16.11 at no inference cost. It tested only many-step image models. We adopt it in the high-band generator and test whether it survives few-step training on temporal skeletons. DiffuPT uses diffusion samples to rebalance glaucoma images. It shows generative rebalancing can help classifiers, but it concerns neither motion nor acceleration.

Research Program

The program has two parts: a measurement study, then a method.

The measurement study trains a many-step flow-matching teacher on each dataset, conditioned on class. On CARE-PD the class is UPDRS-gait score, with site as a nuisance variable. On BEAT it is emotion, plus speech audio and text. We then derive few-step and one-step generators by five routes: naive Euler truncation of the teacher, progressive distillation, Mean-Flow-style average-velocity training, and a DIDiffGes-style adversarial 10-step sampler.

For each generator, class, and number of function evaluations, we compute three things. The first is a per-class Fréchet distance in a motion-encoder feature space. The second is per-class precision and recall. The third is a band-energy ratio

$$R_c(b) = \frac{\mathbb{E}_{\text{gen}}\left[P_c(b)\right]}{\mathbb{E}_{\text{real}}\left[P_c(b)\right]},$$

where $P_c(b)$ is the power spectral density of joint velocities in frequency band $b$, averaged over samples of class $c$. A generator that preserves the class's spectral signature has $R_c(b) \approx 1$.

The prediction is that $R_c(b)$ in high bands, for tail classes, falls faster with decreasing step count than any other cell of the class × band grid. The motivating mechanism predicts this especially for Parkinsonian tremor at roughly 3–8 Hz, and CARE-PD's frame rate permits such bands. If the grid shows no such interaction, the core claim is false. We would report that, and the method part becomes unnecessary.

The method is a band-split generator. A fixed zero-phase low-pass filter separates each motion clip into a low band $x_L$ and a residual $x_H = x - x_L$. A straight-flow generator produces $x_L$ in one or two steps. A second, much smaller flow network produces $x_H$ conditioned on $x_L$, class, and audio when available. It draws independent noise so that random-phase oscillations are sampled rather than averaged. It is trained with class-balanced sampling and a CORAL contrastive term on its bottleneck, and it receives most of the step budget. Because this network is small and runs on a residual, the total wall-clock time per second of motion can be matched to DIDiffGes's 10-step sampler. We will report that matching explicitly, on the same GPU.

For BEAT, which has no clinical labels, we build long-tailed splits by subsampling emotions at controlled imbalance ratios while keeping speakers balanced. This gives a speech-conditioned testbed where ground-truth rare classes are abundant in the held-out full data. We can therefore evaluate against real rare-class distributions, which is impossible on CARE-PD's tiny score-3 class.

Baselines

The naive reference point is no generation at all. A UPDRS-gait classifier, the MotionAGFormer-style encoder that CARE-PD pretrains, is trained on real data only. We run it three ways: with plain cross-entropy, with random oversampling of the minority classes, and with class-balanced or focal reweighting. Every variant uses the same encoder, splits, and hyperparameter search budget as the augmented runs. I found no reported UPDRS-gait classification number on Lacuna for CARE-PD; the file gives only reconstruction MPJPE. We will therefore re-establish these numbers ourselves under leave-one-site-out evaluation. Beating this baseline means a statistically reliable gain in score-3 recall and macro-F1 on held-out sites without a drop in score-0 specificity.

The strongest existing generator is the DIDiffGes-style semi-implicit adversarial 10-step sampler DIDiffGes. It is the only published fast motion generator built to protect low-amplitude detail. Its reported figure is 4.4 s of motion in about 0.4 s on a V100 at 10 steps. Its exact FGD is not stated in the file, and the closest reported BEAT reference is CaMN's FGD of 123.7 from BEAT. We reimplement it on our splits with the same conditioning, data, GPU-hours, and tuning trials as our method. It gets the same class-balanced sampling, so that any advantage we show comes from band splitting and not from rebalancing. Beating it means tail-class high-band $R_c(b)$ closer to 1 and higher downstream score-3 recall at equal or lower wall-clock time.

The many-step flow-matching teacher is the quality ceiling. It is the Match-TTSG-style architecture Match-TTSG, sampled with many steps. If our few-step model matches it on tail metrics, acceleration has been made free for the tail. A Mean-Flow one-step model Mean Flows represents maximal compression.

Three ablations isolate the key ingredients. The first removes the band split but keeps CORAL and class balancing in a single few-step network; this tests whether contrastive regularization alone suffices. The second keeps the band split but removes CORAL and class balancing in the high-band network; this tests whether frequency separation alone suffices. The third keeps the split but gives the high band only one step; this tests whether the benefit comes from spending steps there.

Why not just…?

Why not just sample the many-step teacher offline, since data augmentation does not need real time? This is the most serious objection, and it may be right for pure augmentation. If the teacher's samples help the classifier as much as ours, then for offline rebalancing acceleration is unnecessary. The contribution then narrows to two settings where latency is real. One is interactive virtual-patient avatars of the kind targeted by the virtual patient dialogue system and MedDiT. The other is on-the-fly augmentation, where fresh samples are drawn every epoch. We will report the teacher-augmented classifier as a direct comparison and state the conclusion either way. The measurement study is still valuable: it tells practitioners which accelerated generators are unsafe for tail classes.

Why not just oversample or reweight the classifier? GTSN already used mirroring augmentation and still struggled on scores 3–4. CORAL argues that rebalancing does not undo representation entanglement. Those are different models, though. The no-generation baselines above settle the question on CARE-PD directly. If they match the augmented runs on held-out sites, generation is not worth its cost for this task.

Why not just add parametric tremor, a 4–6 Hz sinusoid, to mild samples? For isolated resting tremor this could work, and we include it as a physics-inspired augmentation baseline. Severe Parkinsonian gait, however, also involves shuffling, reduced arm swing, and freezing episodes that are not additive sinusoids. The comparison will show whether a learned high band captures structure a fixed oscillator misses. The test is whether it beats the sinusoid on per-class Fréchet distance and on downstream recall, not merely on band energy, which the sinusoid can trivially match.

Why not just use a GAN, which is one step by construction? The adversarial 10-step baseline already represents the best adversarial motion generator in the file. DIDiffGes reports that holistic adversarial modeling neglects subtle finger motion, and its authors note training fragility. We will record training failures across seeds for this baseline and for our method, rather than reporting only the best run.

Proposed Experiments and Decisive Checks

The first check is a precondition on real data alone: do severe CARE-PD classes, and rare BEAT emotions, differ from common ones in high-band velocity spectra? If the real spectra do not differ by class, the frequency-band mechanism has nothing to act on for that dataset. We would then restrict the claim to class-level feature borrowing.

The second check is the step-count × class × band interaction from the measurement study. It is decisive for the core claim if tail high-band $R_c(b)$ degrades significantly faster than head or low-band cells as steps drop from many to one.

The third check is a recognizability test. A severity classifier trained only on real CARE-PD data labels synthetic score-3 walks. The fraction labeled 3 rather than 1 or 2 directly measures whether "severe" samples look mild. The same test applies to BEAT emotions, where held-out real rare-emotion clips give the true target distribution, and we additionally report FGD against them.

The fourth check is downstream. Leave-one-site-out UPDRS-gait classification is run with each generator's samples added at matched counts, reporting score-3 recall, macro-F1, and score-0 specificity. For every generator we also report wall-clock latency per second of motion on the same GPU.

The method succeeds if it lies on a better latency–tail-fidelity frontier than the adversarial 10-step sampler and Mean Flows, and if its downstream gain survives the offline-teacher comparison in at least the on-the-fly augmentation setting.