Text-to-3D and image-to-3D systems often start by asking a multi-view diffusion model to paint the same object from several camera angles at once. A diffusion model is a generator that turns random noise into an image by removing noise step by step. The views are then fused into a 3D model. When those views disagree, the result shows the Janus problem: a dog with a second face on the back of its head, a horse with two heads, limbs that shift between views. Today the only reliable way to catch this is to finish sampling every view, reconstruct a 3D model, re-render it, and measure how far the re-renders drift from the generated images. Carve3D's Multi-view Reconstruction Consistency score (MRC) works this way. It is accurate but slow, and it only answers after all the sampling compute has been spent.
We propose to attach a small uncertainty-aware prediction head to the diffusion network. The head reads the model's internal cross-view features at an early denoising step and forecasts, in the same forward pass, how inconsistent the finished views will be. It also reports how sure it is of that forecast. The system can then stop bad samples early, keep promising ones, and keep denoising when it cannot yet tell. This turns an expensive after-the-fact check into a cheap in-flight decision. The uncertainty estimate is what makes the decision safe, because early in denoising much of the outcome is still undecided.
Core Claim
A multi-view diffusion model settles most of the coarse structure that determines a Janus failure, such as where faces and heads go, in its early steps. That structure is visible in its cross-view attention features before it becomes legible in pixels. A lightweight head trained to predict the final MRC can therefore rank trajectories well at early steps. The head outputs a mean and a variance, using the Normal-Inverse-Gamma parameterization from evidential deep learning (a method in which one forward pass outputs the parameters of a distribution over the prediction rather than a single value). Under a fixed compute budget, a rule that acts on this predictive distribution should beat both a point-estimate predictor and pixel-level early checks.
The variance has a concrete meaning here. At step $t$ the final MRC is genuinely random, because it depends on noise that has not yet been drawn. That part is aleatoric uncertainty. Unfamiliar prompts and camera setups add epistemic uncertainty. What is new is the target. Prior single-pass uncertainty work predicts uncertainty about the network's current output. We predict uncertainty about a downstream 3D consistency outcome of a trajectory that is not yet finished.
Position Relative to Prior Work
Our target signal comes from Carve3D. It defines MRC by running a feed-forward reconstruction model on the generated views, re-rendering, and computing LPIPS (a learned perceptual image distance), then uses MRC as a reinforcement-learning reward. On the DreamFusion 415-prompt set it lowers average MRC from 0.0892 (Instant3D-10K) to 0.0606. The authors themselves flag the cost of the reconstruction loop. We keep their metric as ground truth and amortize it. Our approach is also complementary: the head can sit on top of a Carve3D-tuned model.
Two papers motivate the problem without solving it. ERGO down-weights inconsistent synthesized views while fitting Gaussian splats, splitting the loss into excess risk and Bayes error. That estimate is formed afterwards, from optimization gradients, and it cannot tell texture noise from a structural Janus contradiction. WarpRF measures uncertainty by warping depth and color across views of an already-fitted radiance field. It needs finished geometry, and depth warping is blind to a second face that is locally consistent in depth, a limitation echoed by Understanding Pure CLIP Guidance for Voxel Grid NeRF Models. PH-Dropout estimates epistemic uncertainty for view synthesis, but it needs many stochastic passes and targets reconstruction rather than generation.
Several papers supply the method. The heteroscedastic mean-and-variance head follows Mashrur et al.. The Normal-Inverse-Gamma variant follows the regression side of A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation. The same survey also constrains the design. It documents feature collapse, where unfamiliar inputs land in familiar feature regions and get false confidence, and it documents sensitivity to the regularization weight. Both push us to test the method on out-of-distribution prompts and to compare against a feature-density detector in the style of MAPLE.
The backbones are MVDream for text-to-3D and SyncDreamer for image-to-3D. Both already reduce Janus artifacts through cross-view attention or a shared 3D feature volume, but neither reports its own reliability.
Research Program
We first build a labeled corpus. For each of many prompts from a training pool disjoint from the DreamFusion 415 set, we draw several seeds from a frozen MVDream. For each trajectory we cache intermediate features at a small set of denoising steps: the cross-view attention outputs and middle-block activations, pooled per view. We then finish sampling and compute MRC with a feed-forward reconstructor, following Carve3D's recipe. Where the reconstructor supports it, we also compute a per-view re-render error, so the head can report which view is at fault. We repeat this for SyncDreamer on Objaverse-derived inputs. On a held-out subset, human annotators mark Janus failures such as duplicated faces or heads. MRC is only a proxy for these failures, and we need to know whether the head learns the proxy or the phenomenon.
The head is a few transformer layers over the per-view feature tokens, conditioned on the step index. It outputs Normal-Inverse-Gamma parameters for log-MRC and per-view error. The diffusion model stays frozen, so generation quality is unchanged by construction. We will quantify the head's cost relative to one denoising step.
We then define a decision rule. At a checkpoint step, the system stops a trajectory if the predicted probability that its MRC exceeds a threshold is high. It keeps the trajectory if that probability is low. Otherwise it continues to the next checkpoint. Stopped budget is spent on new seeds. We sweep the thresholds to trace a curve of compute against final quality.
Baselines
The naive reference is a single sample per prompt with no filtering, which gives the base model's MRC and Janus rate. Alongside it we run uniform best-of-N at the same total number of network evaluations as our method. It uses random selection when no verifier is available, and full-MRC selection when one is. Full-MRC best-of-N is the accuracy ceiling for filtering, because it uses the true label. Its extra cost is complete sampling plus reconstruction for every candidate. Our method beats it if it reaches the same final MRC with substantially fewer network evaluations.
The strongest early-verification competitor is x0-preview MRC. A diffusion model can estimate the clean image at any step; this baseline feeds those estimated views to the same reconstructor at the same checkpoint step and computes MRC, with no learned head. It uses the same information budget in time, and it is the obvious thing a reviewer would try. We give it identical checkpoints and tune its stopping threshold on the same validation split. Beating it means higher AUROC for predicting final MRC above threshold, and for predicting human Janus labels, at early steps. It also means a better compute-quality curve, without paying the reconstructor's cost at each checkpoint.
A training-free single-pass baseline fits class-free Gaussian densities to the same pooled features on the training trajectories, in the manner of MAPLE and Zhu et al.. It uses Mahalanobis distance as the score. This tests whether plain out-of-distribution detection is enough, or whether training against the outcome is needed.
As the reference for what training-time fixes achieve, we compare against Carve3D: average MRC of 0.0606 on DreamFusion 415, against 0.0892 for its base. That number comes from a different base model (Instant3D). We therefore report relative MRC reduction over each respective base, and we avoid claiming a direct win unless we reproduce on a shared base.
The key ablation replaces the Normal-Inverse-Gamma head with a point regressor of identical size, trained with mean squared error and used with a single threshold. If the uncertainty-aware three-way rule does not dominate the point regressor's curve, the uncertainty component adds nothing and we will say so. A second ablation feeds the head only non-attention features, to test the claim that cross-view attention carries the early signal.
Why not just…?
Why not just generate several seeds and measure their disagreement? Disagreement across seeds reflects how diverse the prompt's outputs are, not whether any single sample is internally consistent. A prompt can yield ten varied but consistent dogs, or ten nearly identical two-faced ones. Disagreement also costs K full samples, the linear scaling that single-pass methods exist to avoid. Evidential TBraTS reports 0.015 versus 3.258 minutes per sample against an ensemble. We include seed disagreement at matched compute as an extra baseline curve.
Why not just run the reconstructor on the early x0 preview? This is the real threat, and it is our main baseline. Early previews are blurry, so the LPIPS comparison may be dominated by blur rather than inconsistency, while the attention features may already encode where heads go. That is a hypothesis, not a fact. If x0-preview MRC matches our AUROC at every checkpoint, the contribution shrinks to a modest reduction in the reconstructor's cost, and we will report that outcome.
Why not just fine-tune the generator with Carve3D-style reinforcement learning? Fine-tuning shifts the average, but it gives no per-sample warning at test time, and Carve3D's authors report substantial training cost. The two approaches stack. We will run the head on a consistency-tuned model if one is available to us, and otherwise on MVDream, to show that residual failures remain predictable.
Why not just train a camera-pose classifier and flag rear views that look like front views? This catches the canonical face-on-the-back case described in DreamBooth3D. It misses duplicated limbs, texture drift, and failures in the image-to-3D setting. It also still requires finished images. We will train such a classifier on final views as a baseline on the human-labeled Janus subset, which shows how much of the failure mode is pose confusion alone.
Proposed Experiments
For text-to-3D, we use MVDream on the DreamFusion 415 prompts. We report AUROC and precision at fixed recall for predicting final MRC above threshold, and for predicting human Janus labels, at each checkpoint step. We also report the compute-quality curve against all baselines, and the calibration of the predicted MRC intervals, as coverage of nominal 80% and 95% intervals.
For image-to-3D, we use SyncDreamer on Google Scanned Objects. SyncDreamer reports PSNR 20.05 and Chamfer distance 0.0261 on 30 objects. We report the same metrics after early-stopping selection at matched compute, along with prediction AUROC.
A secondary experiment feeds the head's per-view error estimates into Gaussian-splat fitting as view weights. We compare this against uniform weights and against ERGO's weighting on Google Scanned Objects; ERGO reports PSNR 21.37 there. The comparison asks whether uncertainty produced before generation finishes can replace gradient-based weighting computed afterwards.
To probe feature collapse, we build a held-out stress set of articulated and human subjects and unusual camera elevations, the regime where the file reports that current multi-view generators degrade. We test whether predicted variance rises or collapses there.
The project is decisive on two checks. The first is early-step prediction: at roughly the first third of denoising, the head must beat x0-preview MRC in AUROC against human Janus labels. The second is the uncertainty ablation: the three-way rule must strictly dominate the point regressor's compute-quality curve. If both hold, single-pass uncertainty buys something real for Janus mitigation. If either fails, we learn that the early signal lives in pixels or that the uncertainty component adds nothing, and both are informative outcomes.
References
- Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning
- ERGO: Excess-Risk-Guided Optimization for High-Fidelity Monocular 3D Gaussian Splatting
- WarpRF: Multi-View Consistency for Training-Free Uncertainty Quantification and Applications in Radiance Fields
- UNDERSTANDING PURE CLIP GUIDANCE FOR VOXEL GRID NERF MODELS
- PH-Dropout: Practical Epistemic Uncertainty Quantification for View Synthesis
- Robust Neural Regression via Uncertainty Learning
- A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation
- Gaussian Latent Representations for Uncertainty Estimation using Mahalanobis Distance in Deep Classifiers
- MVDream: Multi-view Diffusion for 3D Generation
- SyncDreamer: Generating Multiview-consistent Images from a Single-view Image
- Fast uncertainty estimates in deep learning interatomic potentials
- TBraTS: Trusted Brain Tumor Segmentation
- DreamBooth3D: Subject-Driven Text-to-3D Generation