Text- and image-to-3D generators often build an object by asking a 2D image generator for many views and then fusing those views into one 3D shape. The best-known failure is the Janus problem: a dog with a face on the back of its head, because the 2D model has mostly seen dogs from the front. This note proposes a different way to see the problem. A Janus view is a view whose content implies the wrong camera. The back of the dog's head "looks like" a front view, so if you registered that image against its siblings you would place it at the front. That reframing turns Janus detection into a question robotics has studied for decades: given some measurements, which directions of a camera's pose do they actually pin down? Robotics calls the case where they pin down almost nothing geometric degeneracy. A smooth tunnel constrains nothing along its length, and a vase constrains nothing about rotation around its axis.
We propose a consistency score that checks each generated view against the pose it was commanded to depict. The score counts disagreement only along directions the view's content can actually constrain. Shape alone cannot constrain rotation of a symmetric object, so we add appearance as a second measurement there. Per-view depth from a single image is only known up to scale, so scale is treated as an explicit, possibly degenerate, extra direction. The result is a cheap, interpretable signal. It can tell "this object has two faces" apart from "this object is a cylinder and nothing is wrong." We use it as a detector, as a reward for fine-tuning, and as a test-time filter.
Core Claim
Current consistency signals cannot separate hallucination from ill-conditioning. Carve3D's multi-view reconstruction consistency (MRC) reconstructs generated views into a 3D field, re-renders them, and measures the difference. It works well as a reward, but the research file notes it cannot tell whether a high discrepancy comes from Janus hallucination or from a shape whose reconstruction is fundamentally ill-conditioned (Carve3D). MVD-Fusion generates multi-view color-plus-depth images and uses intermediate depth to reproject features between views. Its authors concede that the consistency this produces is a learned tendency, not a guarantee (MVD-Fusion).
Our claim is that a pose-discrepancy test can do better than either signal on symmetric and textureless objects. The test is weighted by the information matrix of a joint shape-plus-appearance registration over seven degrees of freedom: rotation, translation, and scale. It should detect Janus artifacts more accurately, and a reward built on it should cut Janus rates without the reconstruction-in-the-loop cost of MRC. The key ingredient is the degeneracy weighting. If an unweighted pose check does as well, the claim is false.
Concretely, for generated view $i$ with commanded camera $T_i$, let $\hat T_i$ be the pose found by registering view $i$'s depth and appearance against the other generated views. Let $\xi_i$ be the 7-dimensional discrepancy between $\hat T_i$ and $T_i$. Let $H_i = J_g^\top J_g + \lambda J_p^\top J_p$ be the registration information matrix at the commanded pose. Here $J_g$ and $J_p$ are the Jacobians of the geometric and photometric residuals, and $\lambda$ balances the two. The per-view score is
$$ s_i = \xi_i^\top H_i \, \xi_i , $$
which is a Mahalanobis distance: pose disagreement measured in units of how well the content constrains that direction. Large disagreement along a direction the content barely constrains, such as rotation about a vase's axis, contributes little. Disagreement along a well-constrained direction, such as yaw of an animal head with eyes, contributes a lot.
Research Program
The method borrows its pieces from three places. It reuses the eigen-analysis of the registration Hessian, the matrix of second derivatives of the alignment error, from LiDAR odometry. X-ICP splits pose directions into localizable, partially localizable, and non-localizable from this matrix. LION monitors its condition number to detect tunnel "slip." Both freeze degenerate directions. We instead use the spectrum to decide how much a discrepancy should count.
The method also reuses the idea of constraining degenerate directions with a second modality. Intensity-SLAM adds laser reflectivity edges where geometry is uniform. GeGS-PCR shows that naive color concatenation is fragile and that a joint geometric-photometric objective is needed. Our appearance term plays the same role: it is the only thing that can catch a painted-on face on a rotationally symmetric head.
The scale direction comes from monocular depth ambiguity. Per-view depth predicted from a single image is defined only up to an unknown scale along each ray (Towards Ambiguity-Free Spatial Foundation Model). We therefore register in similarity space rather than rigid space, and scale gets its own eigen-direction.
Two engineering choices matter. First, a 180° Janus flip is far outside the range where the local linearization of X-ICP is valid; the file names fragility under poor initialization as its known limitation. We therefore compute $\hat T_i$ by global hypothesis testing over a coarse grid of orbit poses. We then refine locally and evaluate $H_i$ only at the commanded pose. Second, depth comes from the generator when it provides one, as MVD-Fusion does. Otherwise it comes from a frozen monocular depth model applied per view. This makes the test usable on RGB-only multi-view generators such as the Instant3D model used by Carve3D.
The program has three stages. The first is a detection study: does $s_i$ separate Janus from non-Janus view sets better than MRC and simpler checks, especially on symmetric objects? The second is a reward study: following Carve3D's on-policy fine-tuning protocol, we replace the MRC reward with $\sum_i s_i$, which needs no 3D reconstruction per rollout. The third is a test-time study: we use the score to reject and resample individual views of an MVD-Fusion generation, without fine-tuning.
Baselines
The naive reference point is the base generator with no consistency signal: the Instant3D model in Carve3D's setting, and MVD-Fusion with its standard depth-guided attention in the feed-forward setting. Each is run with the same seeds and prompts as every other condition.
The strongest existing competitor on the reward question is Carve3D's MRC-based fine-tuning (Carve3D). It is the only method in the file that directly optimizes multi-view consistency of a diffusion generator and reports a Janus-relevant number. On the 415-prompt DreamFusion test set it reports MRC of 0.0606, against 0.0892 for Instant3D-10K, and a 68.33% human preference for consistency. We will run it with the same base model, the same training prompts, the same number of policy updates, and the same KL regularization strength, changing only the reward. We will also report wall-clock reward cost, since ours avoids fitting a reconstruction per sample.
Beating Carve3D means three things together. Our method must reach an MRC no worse than 0.0606 on that prompt set. Note that this metric is Carve3D's own and favors it. Our method must also achieve a significantly lower human-labeled Janus rate. And it must do so at lower reward compute.
For the feed-forward setting, the strongest competitor is MVD-Fusion, which reports Chamfer distance 0.031 on Google Scanned Objects (GSO) and novel-view PSNR 21.19 dB on Objaverse (MVD-Fusion). Beating it means a lower Janus rate after our test-time view rejection, with Chamfer distance not worse than 0.031.
Two ablations isolate the key ingredient. The first sets $H_i$ to the identity, giving a plain pose-discrepancy check. The second drops the photometric term ($\lambda = 0$), giving geometry-only weighting. A third ablation, rigid instead of similarity registration, tests whether modeling scale matters.
Why not just…?
Why not just use Carve3D's MRC reward? It already works, and on its own metric it is the bar to clear. The open issue is diagnostic. MRC is an image-space re-rendering error. On a textureless sphere or a vase, reconstruction is ill-conditioned and MRC can be high or low independent of any hallucination. Conversely, a consistently hallucinated second face can be reconstructed into a coherent but wrong 3D object with low MRC. Our detection study measures this directly. We will compare detection accuracy of MRC against our score on a split restricted to symmetric and textureless objects. If MRC matches us there, the diagnostic claim fails and we fall back to arguing only compute cost.
Why not just run an off-the-shelf pose estimator on the generated views and flag mismatches with the commanded cameras? This is the most serious threat, because it is the unweighted version of our idea. A feed-forward geometry model of the kind IncVGGT builds on could provide the poses cheaply. The expected weakness is false positives. On symmetric objects, any pose estimator returns an arbitrary rotation about the symmetry axis, and an unweighted check will flag correct generations. IncVGGT itself reports that registration breaks down in featureless, degenerate conditions. Our identity-$H$ ablation is exactly this alternative with our registrar substituted. We will also run it with an external estimator. If either matches the weighted score's precision on symmetric objects, we will drop the weighting claim.
Why not just scale up 3D training data or use depth-guided attention? Carve3D reports that 10× more supervised 3D data (Instant3D-100K) still left substantial inconsistency. MVD-Fusion's authors describe their reprojection as an inductive bias rather than a guarantee. Neither result rules out that a larger depth-aware generator would close the gap. Our feed-forward experiment therefore treats MVD-Fusion as the baseline to improve on, not as a straw man.
Why not just ask a vision-language model whether a view shows a back or a front? Per-frame semantic scoring is precisely what Understanding Pure CLIP Guidance for Voxel Grid NeRF Models found to optimize each view's local plausibility while ignoring global shape. A face on the back of a head is a plausible face. We will include a CLIP front/back classifier as a detection baseline to confirm this on our labeled set.
Proposed Experiments
The first experiment is a decisive check on synthetic data. We render GSO objects from orbit cameras and build controlled Janus cases by copying a front-view texture onto the back geometry. We also keep a matched set of rotationally symmetric and textureless objects with no injected artifact. Detection is measured by area under the ROC curve and by precision at fixed recall, reported separately for asymmetric and symmetric objects. The prediction is specific. The identity-$H$ check and MRC should lose most of their precision on the symmetric split. The full score should not. And the geometry-only ablation should miss painted-on faces on symmetric heads. If the full score does not beat the identity-$H$ ablation on the symmetric split, the central claim fails.
The second experiment uses real generations. We label Janus presence by human annotation on outputs from the DreamFusion 415-prompt set and on MVD-Fusion outputs for GSO and CO3D inputs. We report detection metrics against all detection baselines.
The third experiment is the reward study under Carve3D's protocol. We report MRC, labeled Janus rate, human preference, and reward compute per sample, against Carve3D's reported MRC of 0.0606.
The fourth experiment is the test-time study on MVD-Fusion. Generated views whose score exceeds a threshold calibrated on the synthetic set are resampled. We report Janus rate, GSO Chamfer distance against the reported 0.031, and Objaverse PSNR against the reported 21.19 dB.
Across all experiments we will report the eigenvalue spectra of $H_i$ by object category. If the scale direction is consistently the weakest, that would directly link single-image depth ambiguity to where Janus errors are allowed to hide.
References
- Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning
- MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation
- X-ICP: Localizability-Aware LiDAR Registration for Robust Localization in Extreme Environments
- LION: Lidar-Inertial Observability-Aware Navigator for Vision-Denied Environments
- Intensity-SLAM: Intensity Assisted Localization and Mapping for Large Scale Environment
- GeGS-PCR: Fast and Robust Color 3D Point Cloud Registration with Two-Stage Geometric-3DGS Fusion
- Multi-Hypothesis Spatial Foundation Model: Rethinking and Decoupling Depth Ambiguity via Laplacian Visual Prompting
- IncVGGT: Incremental VGGT for Memory-Bounded Long-Range 3D Reconstruction
- UNDERSTANDING PURE CLIP GUIDANCE FOR VOXEL GRID NERF MODELS