Blur as a Pose Trajectory: Latent-Manifold Articulated Gaussians for Blurry Human Video

When a person moves fast in front of a handheld camera, each video frame records a smear. The smear is the image averaged over every body pose the person passed through while the shutter was open. Current avatar methods based on 3D Gaussian splatting treat this smear as damage. 3D Gaussian splatting represents a scene as many small colored, semi-transparent 3D ellipsoids that can be rendered fast and fitted by gradient descent. Articulated versions attach these ellipsoids to a skinned body skeleton. The damage view fails because the pose detectors that initialize those avatars break on blurred frames. Methods that do model blur for dynamic scenes let the moving parts deform freely, and a free deformation can match a smear with physically impossible stretched or split limbs. This note proposes the opposite reading. The blur in each frame is treated as evidence of a short motion, and that motion is required to be a smooth path inside a learned space of valid human poses. The Gaussian avatar is rendered at several instants along the path, the renders are averaged the way a sensor does, and the result is matched to the blurry frame.

Stated in two sentences: we represent the within-exposure motion of a person as a short curve in the latent space of a pose prior (VPoser, a variational autoencoder whose latent codes decode only to plausible body poses), jointly with a spline for camera shake. We then fit a skinned Gaussian avatar by requiring that the sensor-averaged renders along both curves reproduce the blurry video. Every intermediate pose the optimizer can reach is anatomically valid. That removes the main degeneracy of blur-aware dynamic reconstruction, and the same fit also returns sharp sub-frame 3D poses.

Core Claim

Two pieces of prior work each solve half of this problem. Human from Blur showed that blur can be inverted into human motion. It fits SMPL (the standard skinned body mesh) with polynomial joint trajectories inside the exposure, renders sub-frames with a mesh rasterizer, averages them, and regularizes with an adversarial motion prior trained on motion capture. It assumes a static camera and a matted background, it renders a bare mesh, and it is slow. BAD-Gaussians showed that the same sub-frame averaging works inside Gaussian splatting for camera shake, with B-spline camera trajectories. It assumes a rigid scene.

The dynamic-scene successors BARD-GS and UPGS handle moving content. BARD-GS uses a time-conditioned deformation network and UPGS uses per-ellipsoid rigid warps. Neither has any skeletal or anatomical constraint. BARD-GS itself needed pseudo-3D tracks from an off-the-shelf 2D tracker and monocular depth to stop moving Gaussians from vanishing or popping, and those auxiliaries degrade under exactly the blur being inverted.

E-Sem3DGS is the closest work. It builds skinned human Gaussians plus a background from blurry monocular video, but it depends on an event camera for motion supervision and uses raw SMPL poses with no learned prior. Our claim is that an ordinary RGB camera is enough if the intra-exposure motion is confined to a learned pose manifold. Under that confinement, articulated Gaussian avatars can be recovered from blurry, handheld RGB video with higher sub-frame pose accuracy and fewer limb artifacts than unconstrained deformation. The confinement also works without event data or sharp-frame initialization.

Research Program

Representation. The avatar follows GauHuman. Gaussians live in a canonical body pose and are warped into each posed configuration by linear blend skinning, which moves each point by a weighted mix of its nearby bones. Small networks refine the pose and the skinning weights. GauHuman is our method source for this component, and its fast convergence on sharp video is the property we want to preserve. The static background is a separate Gaussian set. Camera shake during each exposure is a cubic B-spline in camera pose, as in BAD-Gaussians.

Motion model. The new element is the motion model. For frame $k$, the body pose at normalized exposure time $\tau \in [0,1]$ is $\theta_k(\tau) = D(z_k(\tau))$. Here $D$ is the frozen VPoser decoder from Expressive Body Capture, and $z_k(\tau)$ is a quadratic curve in its 32-dimensional latent space. We use a curve rather than a single code so that the start, middle and end of each exposure all decode to valid poses. Curves in adjacent frames are tied by continuity at shared exposure boundaries. That tie resolves the forward-versus-backward ambiguity of a single smear, a problem Human from Blur addressed with its adversarial motion prior.

Sensor model. Low-light capture forces long exposures, so the sensor model matters as much as the motion model. We average renders in linear radiance and apply saturation and a learned per-video response curve only afterwards:

$$ \hat I_k = g\!\left(\frac{1}{N}\sum_{n=1}^{N} \mathcal{R}\big(\mathcal{G}, \theta_k(\tau_n), \mathbf{T}_k(\tau_n)\big)\right) $$

In this equation, $\mathcal{R}$ renders the Gaussians $\mathcal{G}$ at body pose $\theta_k(\tau_n)$ and camera pose $\mathbf{T}_k(\tau_n)$, the $\tau_n$ are $N$ sub-frame instants, and $g$ is a clipping-plus-tone curve with per-frame gain. Averaging after clipping, which is what naive pipelines implicitly do, underestimates bright streaks such as a lit hand sweeping past a dark background. The order therefore matters most in exactly the high-dynamic-range regime where blur is worst.

Initialization. Detectors fail on blur, so we replace them with a blur-aware initializer built from cross-modal features. A small network reads frozen DINOv2 features of the blurred frame and regresses the three control codes of $z_k(\tau)$. It is trained on synthetic blur generated by rendering AMASS motions, the same recipe behind BT-AMASS. This is where feature fusion enters: 2D foundation features are mapped into the pose manifold, not lifted onto 3D points. Naive projection of smeared features onto geometry is exactly what the fusion literature warns against when poses drift. After initialization, optimization is photometric, plus the VPoser latent penalty and GauHuman's canonical regularizers.

Training follows the staged schedule that UPGS found necessary for convergence. The camera spline and background are fit first on frames where the human mask is dilated out. Then the latent trajectories and avatar are fit with the camera frozen, and finally everything is fit jointly.

Baselines

The naive reference point is vanilla GauHuman run directly on the blurry frames with SMPL poses from an off-the-shelf estimator. It measures how much blur costs when nothing is done.

The pre-deblurring reference runs the same GauHuman pipeline on frames sharpened by a 2D video deblurring or interpolation network, with poses re-detected on the sharpened frames.

The strongest RGB-only dynamic competitor is BARD-GS, which reports 25.13 PSNR on its real-world dual-GoPro dynamic blur benchmark, against 22.06 for 4D-GS and 20.57 for DyBluRF. We run it with its own 2D-track and depth losses on every human sequence we test. UPGS is run the same way; on Stereo Blur it reports a 1.12 dB PSNR gain over Deblur4DGS.

The strongest human-specific competitor is E-Sem3DGS. It reports 32.56 PSNR on ZJU-MoCap-Blur, against 21.75 for 3DGS-Avatar, and 28.63 PSNR on MMHPSD-Blur, against 25.23 for HUGS. It uses simulated or real events that our method does not. We therefore compare against its published numbers as an event-equipped upper reference and also rerun it with the event-flow term disabled. For RGB-only human avatar reconstruction under blur, no reported result found on Lacuna, so that rerun is the fair head-to-head. Human from Blur is the pose-accuracy baseline on BT-AMASS and B-AIST++; no numerical MPJPE value is reported in the file, so we rerun its released pipeline.

Fairness rules apply to every learned-deformation competitor. Each receives the same sub-frame count $N$, the same background and camera stage, the same GPU-hour budget, and the same number of hyperparameter trials on a held-out sequence.

Three ablations isolate the key ingredient. In the first, the latent curve is replaced by a quadratic curve directly in SMPL axis-angle space. It keeps skinning, sub-frame averaging and continuity, but drops the learned manifold. The second removes the continuity tie. The third moves clipping before averaging. Beating the baselines means two things at once. On MMHPSD-Blur, sub-frame pose error against ground-truth SMPL must be lower than the axis-angle ablation and BARD-GS, and novel-view PSNR must be at least matched. On sharp MonoCap, GauHuman's 33.45 PSNR must be retained within noise, which shows the prior does not cost fidelity when there is no blur.

Why not just…?

Why not just deblur the frames in 2D and run GauHuman? Frame-wise restorers hallucinate detail that is inconsistent across frames, and 3D fitting turns that inconsistency into floaters; BAD-Gaussians reports its forward-model gains largely against such pipelines. Human from Blur also found that 2D deblurring, which assumes rigid or uniform motion, does not recover articulated joint positions. This is our pre-deblurring baseline, and the sub-frame pose metric on MMHPSD-Blur settles the question directly.

Why not just use an unconstrained deformation field as in BARD-GS or UPGS? Such fields can fit a smear with sheared or split limbs, and BARD-GS already needed external tracks to prevent Gaussians from vanishing. We expect photometric scores to be close on mild blur. Our claim is specifically about severe blur and pose correctness, so we stratify results by blur magnitude and report limb-length variance and joint-limit violations across sub-frames. If the unconstrained fields match us on those measures, the claim fails.

Why not just constrain motion with SMPL skinning alone, without a learned manifold? This is the most serious threat. Skinning already forbids stretching bones, and E-Sem3DGS works without a learned prior. What skinning does not forbid is hyperextended joints, and it does not smooth the loss landscape; the Expressive Body Capture paper argues that a learned prior is what smooths it. The axis-angle ablation is designed to answer this question exactly. If it matches the latent curve on pose error at high blur, we will report that the manifold is unnecessary and reframe the contribution around the sensor model and the blur-aware initializer.

Why not just add an event camera? Events do help, as E-Sem3DGS shows. They require special hardware, however, and the file notes their noise and synchronization problems in low light. We treat events as an optional extra term, not a dependency, and measure how much of E-Sem3DGS's event advantage the manifold recovers from RGB alone.

Proposed Experiments

The primary benchmarks are MMHPSD-Blur and BT-AMASS. On these we report sub-frame MPJPE (mean per-joint position error), anatomical-violation rates and novel-view PSNR/LPIPS, because both have ground-truth body parameters. For avatar quality we use ZJU-MoCap-Blur and add camera shake synthetically to create a handheld variant. For real footage we use B-AIST++ and the 12-scene BARD-GS real benchmark; on the latter we evaluate only scenes containing people and report the human-masked PSNR separately. The dynamic-range test uses the 21 real low-light sequences from Human from Blur where available. We supplement them with synthetic renders whose highlights are clipped, and compare averaging before versus after clipping.

Three checks are decisive. The first is the curve of pose error against blur magnitude for the latent path, the axis-angle path and BARD-GS; the gap should widen as blur grows if the manifold is doing the work. The second is whether reversing time in the recovered trajectories raises the photometric loss once continuity ties are on, which tests whether the direction ambiguity is actually resolved. The third is wall-clock time to convergence against GauHuman on sharp video and against UPGS's reported 4.2 hours on blurry video.