Containing Dataset Corrections in Camera-Controlled Video Models

When an open dataset issues a correction, every model trained on it inherits the error, and the only remedy anyone uses today is to retrain from scratch. That is the machine-learning version of a global cluster restart after a node crash. Large-scale training systems stopped accepting that cost years ago: they recover only the state that was actually damaged. We propose to do the same for data errors. A training run logs which data records, and which fields within them, each step consumed. A correction is published as a machine-readable list of the records and fields it changes. A recovery planner then restores and retrains only the part of the model that read the bad field, starting from the last checkpoint taken before the bad data was first seen.

Video generators that control camera motion and subject motion separately are where this becomes practical and where it matters most. In these models the camera signal passes through its own module, so a correction to camera labels has a natural place to stop. Camera labels for real footage also come from automatic estimators that fail in predictable ways on scenes with moving people. The core claim is twofold. First, for camera-conditioned video diffusion transformers, errors in camera labels do not wash out under continued training: the model learns to distrust the camera input and drifts toward static output. Second, rolling back only the camera module to a checkpoint from before the error, then retraining it on the corrected data, recovers retrain-from-scratch quality at a small fraction of the compute. Doing this requires architectural decoupling plus a training-step data log. Neither alone is enough.

Why this is the right intersection

Correcting the scientific record is slow and manual. A corrigendum (a formal published correction) can fix something as simple as two swapped figure labels, as in the haplotype correction of Bevers et al.. During the "window of risk" before that notice appears, downstream work builds on the error. Over 30% of retraction notices do not even say what was wrong (Cheng et al.). Treat societally impactful scientific insights as open-source software artifacts argues for treating scientific claims like versioned code with automated tests, but it tested this only on student projects. These works motivate the problem. None of them says what a correction should do to a trained model.

Dataset transparency has moved in the right direction. HumanVid released 20K real clips whose camera paths come from SLAM (simultaneous localization and mapping, which estimates camera pose from video) and whose human poses come from a 2D pose estimator. It also released 75K synthetic clips with exact camera ground truth. TokenMotion trains on HumanVid. The research file notes that HumanVid offers no mechanism to version or correct its noisy real-world labels after release. The failure that matters here is specific. When a person walks through the frame, SLAM can attribute some of that subject motion to the camera. A model trained on such labels learns exactly the camera–subject entanglement that decoupled architectures are built to avoid.

Fault-tolerant training supplies the method. SCAR treats a hardware failure as a bounded perturbation, recovers only the lost parameters, and checkpoints the fastest-changing parameters first. It cut rework by 78–95% relative to full checkpoint restore. Maeng et al. restore only the failed embedding tables in recommendation models. Training Overhead Ratio measures reliability as ideal time over observed time and accounts separately for rollback waste and repair time. We reuse all three ideas, but the "failure" is a data erratum rather than a crashed node.

One citation constrains the design. SCAR's guarantees assume perturbations wash out under iterative optimization. Cheong et al. show that video diffusion transformers dilute camera conditioning and fall back toward static output. VD3D found it necessary to freeze the backbone and inject camera rays (Plücker embeddings, one 3D ray per pixel) through a zero-initialized side module. If the wash-out assumption fails for camera control, continued fine-tuning is a weak remedy. VD3D's design also tells us where the damage lives: in the camera side module.

Core Claim

A decoupled camera-and-subject video model, trained with a per-step log of data consumption, can absorb a published correction to one annotation field by restoring and retraining only the module that consumes that field. The training restarts from the last checkpoint before the erroneous records were first seen. The claim has three testable parts:

  • Persistence. Camera-label errors leave a lasting deficit in how much the model responds to its camera input.
  • Locality. That deficit sits mostly in the camera module, not in the backbone or the pose module.
  • Advantage. Lineage-scoped partial rollback beats compute-matched alternatives.

To make the persistence claim measurable, we define the model's camera gain by analogy with Cheong et al.'s "null camera" baseline. Camera gain is the motion magnitude the model generates under a target trajectory minus the motion magnitude under a static camera, at fixed prompt and seed. A model that has learned to distrust its camera input has low camera gain even when its image quality is fine.

Research Program

What gets built. There are three components: a lineage log, an erratum format, and a recovery planner.

The lineage log is written by the data loader. For every optimizer step it records the clip IDs consumed and a hash of each annotation field read: camera trajectory, pose, and caption. The log costs kilobytes per step. Checkpoints of the trainable modules are taken every k steps. These are cheap because the backbone is frozen and only the side modules change.

An erratum is a versioned diff over clip IDs and field names. Its form is a statement such as "camera trajectories of clips S are replaced; pose unchanged." This turns the unstructured correction notice of publishing practice into an object a program can act on.

The recovery planner works in three steps:

  1. It maps each corrected field to the modules that read it. Camera fields go to the camera module; pose fields go to the pose module.
  2. It looks up in the log the first step at which any corrected record was consumed.
  3. It restores those modules alone from the preceding checkpoint and replays the remaining steps with corrected data. The replay is optionally shortened by upweighting the corrected clips.

The model. The test model is a CogVideoX-2B backbone, the backbone TokenMotion used, kept frozen. It carries two trainable modules: a VD3D-style camera module that takes Plücker embeddings, and a pose-guider module in the style of CamAnimate. We also build an entangled control. This variant has the same trainable parameter count, but camera and pose tokens enter one shared conditioning module. It tests whether decoupling is what contains the damage.

The corruption protocol. We need known ground truth, so errors are injected into data whose correct labels we have:

  • Subject-coupled drift. We add to the camera translation a component proportional to the tracked subject's displacement. This mimics SLAM absorbing subject motion. It is applied to the synthetic HumanVid split, where true camera paths exist, and to a fraction of the real split.
  • Convention error. We swap or flip a coordinate axis in a subset of trajectories. This is the data analogue of the swapped figure labels in the haplotype correction.

Corruption rates sweep from 5% to 40% of camera-labelled clips. The time at which the erratum arrives sweeps from 25% to 100% of the training run. The second sweep models the window of risk.

A secondary regime. The same planner handles hardware silent data corruption, meaning hardware that returns wrong numbers without raising an error. In this regime the "erratum" comes from a late-detecting health monitor rather than a dataset maintainer. Ma et al. found gradient noise-to-signal ratios up to 5.1% on unhealthy nodes. They also found that standard checksums (ATTNChecker-style algorithm-based fault tolerance) missed corruption on most of those nodes. We inject gradient noise at that ratio into one simulated data-parallel worker's camera-module gradients for a bounded window, then apply the planner. This tests whether a single mechanism serves both kinds of correction.

The headline efficiency number. By analogy with the Training Overhead Ratio, we define a correction overhead ratio:

$$\text{COR} = \frac{C_{\text{fix}}}{C_{\text{scratch}}}$$

Here $C_{\text{fix}}$ is the GPU-hours a strategy spends after the erratum arrives until it matches the from-scratch model on the quality criteria below, and $C_{\text{scratch}}$ is the GPU-hours of a full retrain on corrected data. A strategy that never reaches the criteria has COR undefined, and we report it as such.

Baselines

Do nothing. The naive reference keeps the exposed model as is. It shows how large the damage actually is. If the damage is small on every metric, the problem is not worth solving, and we will say so.

Retrain from scratch on corrected data. This is the quality ceiling and defines $C_{\text{scratch}}$. It uses the same architecture, data order (apart from the corrected labels), and hyperparameters as the original run.

Strongest external reference points. For camera accuracy on RealEstate10K, VD3D reports translation error 0.409 and rotation error 0.043 (VD3D). For human animation on TikTok, CamAnimate reports PSNR 18.762 and FVD (Fréchet Video Distance, lower is better) 691.8 (HumanVid). Our clean-data model should land near these numbers. Otherwise the corruption study is being run on a weak model and the conclusions do not transfer. No reported result was found on Lacuna for any model trained on corrupted HumanVid camera labels, or for any recovery after a dataset correction. The relevant comparisons are therefore internal and compute-matched.

Strongest competing method: continued fine-tuning. This resumes all trainable modules from the final exposed checkpoint on corrected data. It is cheap, standard, and plausibly sufficient, which makes it the real competitor.

Global rollback and replay. This restores every trainable module from the pre-exposure checkpoint and replays on corrected data. It is the direct analogue of checkpoint restart.

Fairness rules. All strategies get the same post-erratum compute budget. Each is evaluated at budgets of 5%, 10%, 20%, and 40% of $C_{\text{scratch}}$. Each gets the same small learning-rate and upweighting sweep, with three seeds.

What counts as winning. Our method beats continued fine-tuning if it reaches within 5% of the from-scratch model on rotation error, camera gain, and FVD at a lower compute budget. It beats global rollback if it reaches the same quality at lower COR while leaving pose error unchanged.

Ablations that isolate the two ingredients. The first is camera-module-only fine-tuning from the final checkpoint. It uses locality but no lineage. The second is camera-module reset to initialization. It uses locality but discards all pre-exposure learning. The third is partial rollback of the pose module instead, at the same parameter count; it is a placebo that should not help. The last is the entangled control architecture under the same planner. If partial rollback works equally well there, decoupling is not the reason it works.

Why not just…?

Why not just retrain from scratch? For a 2B-parameter backbone with small side modules, that is affordable once. Errata to large open datasets are not one-off events, however, and each downstream group pays the full cost again. Retraining is our quality ceiling, and COR measures how much of its cost is avoidable. If COR for the best cheap strategy stays above roughly 0.5, retraining is the honest recommendation, and we will report that result.

Why not just keep fine-tuning on the corrected data? This is the serious threat. If optimization washes out the error, as SCAR's analysis assumes, continued fine-tuning will match our method, and the lineage log adds little. The opposing evidence is indirect: Cheong et al.'s finding that diffusion transformers drift toward static output when camera conditioning is weak, and VD3D's reliance on frozen backbones. Neither tests label correction directly. The camera-gain trajectory after the erratum settles it. If camera gain under continued fine-tuning recovers to the from-scratch level within the 10% budget across corruption types, the persistence claim is false. The contribution then reduces to the lineage log and erratum format as documentation infrastructure, and the paper will say that plainly.

Why not just filter noisy labels before training, or train only on synthetic clips with exact cameras? Filtering only catches errors someone already knows how to detect. Errata exist precisely because some errors are found after release. Training on synthetic clips alone leaves the visible rendering gap that HumanVid itself reports, and HumanVid's design relies on real clips for texture and variety. As a check, the synthetic-only model is evaluated on TikTok and on the Movable Objects set from I2VControl-Camera. No reported result was found on Lacuna for synthetic-only training on HumanVid. If this model matches the corrected real-plus-synthetic model, the practical case for correcting real labels weakens. The mechanism would still be needed for pose-field and caption errata.

Proposed Experiments

Where the model is evaluated.

  • RealEstate10K measures camera accuracy (translation and rotation error).
  • TikTok measures human animation quality (PSNR, FVD, pose error).
  • VBench++ (Huang et al.) supplies its camera-motion dimension and its optical-flow Dynamic Degree score. These track whether the model is drifting toward static output, the consistency–motion tradeoff VBench++ identifies.
  • Movable Objects (480 clips) probes entanglement. We hold the camera static and animate the subject, and separately pan the camera with a still subject. We then measure how much background motion appears in the first case and how much foreground motion leaks into the second.

Decisive check one: persistence. We plot camera gain and rotation error after the erratum for every strategy at each corruption rate and each exposure time. Persistence holds if continued fine-tuning stays more than 10% below the from-scratch camera gain at the 20% budget. The test is pre-registered.

Decisive check two: locality. Damage should concentrate in the camera module. Swapping only the camera module from the from-scratch model into the exposed model should recover most of the gap; swapping only the pose module should recover little. Pose error should stay flat throughout camera-only errata. If backbone-adjacent effects dominate, so that neither swap recovers the gap, partial rollback cannot work, and we report that.

Decisive check three: the role of decoupling. On the entangled control architecture, scoped rollback should either fail to recover camera gain or degrade pose error. If it does neither, the claim that decoupling bounds the damage is wrong.

Secondary check: hardware corruption. For injected gradient corruption, we compare the same strategies on the same metrics. We also report how quality depends on detection delay, mirroring the exposure-time sweep. If one planner handles both data errata and hardware corruption with similar COR, that supports treating a training record as something correctable after the fact, rather than a static artifact frozen at release.