Recovering a 3D human skeleton from one photo is ambiguous in a specific way. Once you know each bone's 2D image position and its length, the only thing left to decide is whether the bone points toward the camera or away from it. A single image therefore admits a small family of distinct 3D poses that all look the same on screen. Generative models (diffusion models, which learn to turn random noise into samples step by step, and Langevin samplers, which walk downhill on a learned "energy" with added noise) can represent that family and report where it disagrees. Their drawback is speed. They need hundreds of steps per sample and dozens of samples before the spread is trustworthy. When the chains are cut short to save time, they stay near their starting point and report too little uncertainty. The best methods that learn 3D pose without any 3D labels produce a single answer and no uncertainty at all.
We propose to stop asking the sampler to discover the discrete ambiguities, because geometry already lists them. A fast lifting network proposes one pose. We then enumerate the depth-sign flips of the bones it is least sure about, which gives a set of candidate poses. From each candidate we run a short Langevin chain on an energy built entirely from 2D evidence, and we reweight the resulting samples by that energy. The expensive part of sampling, crossing between distant modes, is replaced by an exact geometric move. The chain is left with the part it handles well at short length, which is local refinement. The result is a calibrated set of 3D pose hypotheses, trained and evaluated for calibration without 3D ground truth, at a small fraction of the cost of long-run sampling.
Core Claim
In label-free monocular 3D lifting, the unsupervised training loss is itself a usable posterior over 3D poses. In ElePose that loss combines three things: reprojection agreement with the observed 2D keypoints, the likelihood of the pose re-rendered from random viewpoints under a 2D pose model called a normalizing flow, and soft bone-length priors. ElePose uses it only to train a regressor that outputs one point estimate. We instead evaluate it at test time as an energy $E(y \mid u)$ over 3D joint positions $y$ given 2D keypoints $u$:
$$E(y \mid u) = \lambda_r \lVert \Pi(y) - u \rVert^2 \;-\; \lambda_f \,\mathbb{E}_{R}\big[\log p_{\text{2D}}(\Pi(R\,y))\big] \;+\; \lambda_b\, B(y),$$
Here $\Pi$ is perspective projection, $R$ is a random camera rotation, $p_{\text{2D}}$ is a normalizing flow trained only on 2D poses, $B$ penalizes implausible bone proportions, and the $\lambda$ terms are weights. Sampling $p(y \mid u) \propto \exp(-E(y \mid u))$ needs no 3D labels.
The claim has two parts. First, the modes of this posterior are almost entirely indexed by per-bone depth signs. Seeding short chains at explicitly flipped configurations should therefore recover the posterior spread that long cold-start chains recover, at a fraction of the cost. Second, the spread can be checked for calibration using a held-out camera's 2D keypoints rather than 3D ground truth.
Relation to Prior Work
The case that sample spread is worth having comes from generative uncertainty work outside human pose. Wolleb et al. show that repeated diffusion runs act as an implicit ensemble whose variance traces ambiguous tumor boundaries. The cost is high: 48 seconds per sample over 1,000 steps, with Dice plateauing only after about 20 samples. UnPose uses uncertainty over diffusion-hallucinated object surfaces to weight them against real sensor data. Its authors name uncertainty sampling as the pipeline's main bottleneck. These papers motivate the proposal: spread is useful, and latency is the obstacle.
The sampling machinery we reuse comes from two sources. Nijkamp et al. show that a fixed, short, non-convergent Langevin run of 100 steps behaves as a workable generator. Langevin Predictive Coding warm-starts chains from an amortized encoder and uses adaptive preconditioning. Both papers also constrain us. Short and warm-started chains stay in the basin where they begin, which is exactly the variance collapse that would make uncertainty maps untrustworthy. Our design answers that constraint directly: the starting basins are chosen by enumeration rather than inherited from a single proposal. Both works are also confined to 2D image synthesis and never measure uncertainty calibration. We move the machinery to kinematic 3D geometry and make calibration the primary metric.
The label-free lifting side supplies the energy and the application setting. ElePose, Kundu et al., Zhou et al. and Rhodin et al. reach usable 3D pose from 2D, kinematic or multi-view signals, but all output a single deterministic estimate. Ronchi et al. document the front-to-back flip errors that such estimates make. Those flips are the discrete ambiguity we enumerate.
The closest multi-hypothesis 3D work, PT43D, samples diverse object shapes with an autoregressive transformer. It needs full 3D CAD supervision and pays sequential decoding cost for every sample. We need neither. On the fast side, Trust3R and CFRE produce uncertainty in one forward pass, but each assumes a single-peaked distribution and trains with full 3D targets. We take their speed goal and drop both assumptions.
Research Program
The system has three stages.
Proposal network. An ElePose-style lifter is trained exactly as in the original work. It is extended to output a per-bone flip score, meaning the network's estimated probability that the bone's depth sign is wrong. Because only 2D data are available, the flip-score head is trained self-supervised. For each training pose we compute the energy after flipping each bone. The head learns to predict which flips land in comparably low energy.
Flip enumeration. At test time we take the $m$ highest-scoring bones and form their $2^m$ sign combinations. For each combination we re-solve the child joints' depths while holding 2D positions and bone lengths fixed. This is exact: the relative depth of a bone with length $L$ and projected length $d$ is $\pm\sqrt{L^2 - d^2}$. We expect $m$ between 2 and 4, so between 4 and 16 seeds.
Refinement and reweighting. From each seed we run $K$ preconditioned unadjusted Langevin steps on $E$, with $K$ between 5 and 20. Samples are pooled with self-normalized importance weights proportional to $\exp(-E)$. This lets a seed that lands in an implausible pose, for example a knee bending backward, receive near-zero weight rather than inflating the variance.
The per-joint posterior standard deviation and the weighted mode masses are the uncertainty outputs.
Calibration without 3D labels. Human3.6M and MPI-INF-3DHP are recorded by several synchronized cameras. We lift from one view, project every sample into a held-out camera, and score the observed 2D keypoints there under the projected sample mixture. The scores are negative log-likelihood and interval coverage. This measures whether the predicted 3D spread is right using only 2D annotations. We then check how closely this label-free calibration score agrees with calibration measured against true 3D. That agreement is itself a result: it would tell practitioners whether they can tune uncertainty on in-the-wild multi-view video that has no motion capture.
Baselines
The do-nothing reference is the deterministic ElePose lifter. It gives no uncertainty, so on calibration metrics it is scored with constant per-joint variance fitted on validation data. On accuracy it reports 36.7mm PA-MPJPE on Human3.6M, against 42.0mm for earlier unsupervised methods and 37.1mm for the supervised Martinez et al. baseline, as reported by ElePose. We must not lose accuracy relative to this number. The weighted posterior mean should match or beat 36.7mm, and the best of $S$ samples should beat it clearly, since multiple hypotheses are only useful if one of them is closer to the truth.
The strongest competitor on sample quality is long-run cold-start Langevin on the same energy: 1,000 steps from random initializations, as many chains as the latency budget of our method times twenty. It is the natural "gold" sampler because it targets the same posterior with no seeding shortcut. Beating it means matching its per-joint variance, mode masses and held-out-view negative log-likelihood within its own seed-to-seed noise while using at least ten times fewer energy evaluations.
The strongest single-pass competitors use the same backbone, the same 2D-only data and the same hyperparameter search budget. The first is a heteroscedastic Gaussian head in the style of CFRE. The second is a Student-t evidential head in the style of Trust3R. Neither can use its paper's 3D targets, so both are trained by minimizing the expected energy under the predicted distribution. We also include an ElePose ensemble of five networks and MC dropout, matched to our wall-clock time.
The multimodal single-pass competitors are two. One is a $K$-output mixture head trained with a min-over-hypotheses energy plus learned mode weights. The other is a conditional normalizing flow sampler trained by reverse KL divergence on the energy.
Three ablations isolate the key ingredient. The first removes flip enumeration, so chains start only from the network's proposal. The second keeps flips but drops Langevin, using importance reweighting of the seeds only. The third replaces learned flip scores with random bone selection.
No reported result was found on Lacuna for uncertainty calibration (AUSE, coverage, or negative log-likelihood) of label-free 3D pose methods. Those numbers will be produced here for all baselines under identical protocol.
Why not just…?
Why not just put an evidential or heteroscedastic head on ElePose? It costs one forward pass, and for many joints a single bump is the right shape. The problem is the flipped limb. A single-peaked distribution fitted to a two-way depth ambiguity either centers between the two answers, which is a pose no person could hold, or commits to one and understates its error. Trust3R names this as its core limitation, and Ronchi et al. show such flips are common in monocular lifting. We will report results separately on frames where the long-run sampler finds more than one mode with non-trivial mass. We expect the single-pass heads to be competitive elsewhere. If they are also competitive on these multi-mode frames, our method is unnecessary, and the experiment is designed to show that.
Why not just a mixture head or a conditional flow trained on the energy? This is the most serious threat. Both are single-pass and multimodal. Our concern is that min-over-hypotheses training tends to leave some heads unused or duplicated, and reverse-KL training is known to seek one mode and drop others. The short-run and warm-start results of Nijkamp et al. and Zahid et al. show how easily amortized or truncated samplers stay confined to one mode. Still, a well-tuned mixture head could simply learn the flips. The decisive comparison is mode-mass error against the long-run sampler on multi-mode frames. If the mixture head matches us there at lower latency, we will drop the Langevin stage and report flip-aware mixture training as the contribution.
Why not just ensembles or MC dropout? Ensembles of the same deterministic lifter tend to agree on the same flip, because every member sees the same 2D evidence and the same training objective. Their spread therefore reflects training noise rather than geometric ambiguity. Wolleb et al. also report that dropout-style uncertainty is broader and less localized than generative sample variance. Ensembles additionally multiply backbone cost, the overhead Horseshoe Splatting avoids in radiance fields. Matched-time ensembles are in the baseline set, and their mode-mass error will answer this directly.
Proposed Experiments
We train with 2D keypoints only on Human3.6M and MPI-INF-3DHP. 3D ground truth is used only for evaluation. 3DPW is used for transfer: the VIBE paper reports 51.9mm PA-MPJPE there for a supervised temporal model, and no reported label-free result was found on Lacuna for ElePose on this set. For context on harder settings, Kundu et al. report 89.4 MPJPE on Human3.6M and 97.1 on MPI-INF-3DHP without supervision.
The accuracy metrics are MPJPE and PA-MPJPE of the weighted mean and of the best of $S$ samples. The uncertainty metrics are per-joint AUSE, which asks whether predicted spread ranks errors correctly; 90% interval coverage; and flip rate. The label-free metric is held-out-camera 2D negative log-likelihood. We report latency both as energy evaluations and as milliseconds per sample set.
Three checks decide the project.
The first is a latency–variance curve. We plot mode-mass error and per-joint variance ratio relative to the long-run sampler as the number of Langevin steps $K$ grows. The prediction is that flip-seeded chains reach the long-run values at small $K$, while unseeded chains stay collapsed at every short $K$. This would be a direct resolution of the tension between truncated sampling and faithful uncertainty.
The second is the rank correlation, across all methods and checkpoints, between held-out-view calibration and true 3D calibration. A high correlation would license tuning in the wild without motion capture.
The third is the multi-mode-frame comparison against the mixture head. It decides whether the sampler is needed at all or whether flip-aware amortization alone suffices.
Failure on the first check, meaning seeded short chains also collapse, would falsify the central claim that pose-lifting modes are sign-indexed.
References
- ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Poses
- Diffusion Models for Implicit Image Segmentation Ensembles
- UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimation
- Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model
- Sample as you Infer: Predictive Coding with Langevin Dynamics
- Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation
- Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach
- Learning Monocular 3D Human Pose Estimation From Multi-View Images
- It's all Relative: Monocular 3D Human Pose Estimation from Weakly Supervised Data
- PT43D: A Probabilistic Transformer for Generating 3D Shapes from Single Highly-Ambiguous RGB Images
- Trust It or Not: Evidential Uncertainty for Feed-Forward 3D Reconstruction with Trust3R
- Continuous Normalizing Flows for Uncertainty-Aware Human Pose Estimation
- Horseshoe Splatting: Handling Structural Sparsity for Uncertainty-Aware Gaussian-Splatting Radiance Field Rendering