The Janus Problem as a Mixed-Measurement Problem

Text-to-3D systems often build a 3D object by repeatedly rendering it from a random camera and asking a 2D image generator, trained only on flat pictures, whether each render looks right. Score distillation sampling is the standard version of this: a 2D diffusion model, a generator that learns to turn random noise into images step by step, supplies a gradient that nudges each render toward what it considers a likely image. The best-known failure is the Janus problem, where the object comes out with a face on the back as well as the front. We propose treating this failure as a measurement-noise problem. When the 2D model is asked for "a dog, back view," what it actually returns is a mixture: mostly back views, plus a sizeable share of front views, because its training photos are dominated by frontal shots. Current pipelines assume each camera query is a clean measurement of the requested view. A single object forced to agree with every one of these mixed answers ends up with front features spread onto the back.

Quantum state tomography dealt with the same mismatch long ago. Protocols built and validated on pure states, meaning noiseless and maximally specified states, break on real hardware, where the measured states are statistical mixtures (Experimental neural network enhanced quantum tomography). The standard fix is to put the noise channel into the forward model and estimate it jointly with the state (Universal framework for simultaneous tomography of quantum states and SPAM noise), rather than pretending measurements are clean. We carry that discipline over. We measure the 2D prior's viewpoint confusion as an explicit channel. We then route each distillation gradient to the camera the prior's answer actually depicts, instead of the camera we asked about. The claim is that this correction removes most Janus artifacts at test time, without fine-tuning, a multimodal critic, or a reconstruction model in the loop. A second claim is that the shape of the measured channel predicts which prompts will fail.

Core Claim

Let $\theta$ be the requested camera azimuth and $\phi$ the azimuth that an image from the prior actually shows. The prior's view-conditioned distribution behaves as

$$p_{\text{prior}}(x \mid \theta) \approx \sum_{\phi} K(\phi \mid \theta)\, p(x \mid \phi),$$

where $x$ is an image, $p(x \mid \phi)$ is the distribution of correct images at azimuth $\phi$, and $K(\phi \mid \theta)$ is a viewpoint confusion kernel. Each row of $K$ sums to one. Standard score distillation implicitly assumes $K$ is the identity. The analogy to quantum noise channels makes a sharp, testable prediction.

When $K$ mixes views uniformly, which is the analogue of depolarizing noise, the frontal-feature pull is the same at every camera and mostly rescales the gradient. Neural-Shadow Quantum State Tomography reports the quantum version of this: under local depolarizing and amplitude-damping noise, measurement noise does not change the direction of the optimizer's gradient. Real priors are not uniform, though. Their kernel is biased toward frontal views, much as amplitude damping pulls every state toward a single ground state. Our contention is that this bias is what moves the optimum toward multi-faced objects. The fix is channel-aware distillation. At each step, we form the prior's denoised estimate $\hat{x}$ from the render at $\theta$ and compute a responsibility $R(\phi \mid \theta, \hat{x}) \propto K(\phi \mid \theta)\,s(\hat{x}, \phi)$. Here $s$ scores how well $\hat{x}$ matches the current object's render at $\phi$, or a pose estimator's likelihood for $\phi$. We then apply the distillation gradient to renders at $\phi$ with weight $R$:

$$g(\theta) = \sum_{\phi} R(\phi \mid \theta, \hat{x})\; \nabla_{\text{asset}}\, \mathcal{L}_{\text{SDS}}\big(\text{render}(\phi), \hat{x}\big).$$

A hallucinated face produced when we ask for the back view is therefore sent to the front, where it reinforces the correct face instead of creating a second one.

Research Program

The work has three parts. The first is channel estimation. For each prompt class, we sample the 2D prior under each view prompt and label the true azimuth of each sample with an off-the-shelf pose estimator. We first check that estimator on Google Scanned Objects renders with known poses, using the objects from the evaluation set of Hallo3D. This produces an empirical $K$ per prompt class, and we report how biased and how ill-conditioned it is. The second part is the routed distillation update above, plugged into a standard radiance-field text-to-3D pipeline. It adds one pose-scoring call per step and no extra diffusion calls. The third part is analysis. On toy objects with a known $K$, we test whether plain score distillation converges to front-feature-duplicated optima exactly when $K$ is asymmetric, and whether routing recovers the true object.

The quantum literature also predicts where the method should break. Channel inversion becomes ill-posed when noise dominates the signal. Streaming quantum state purification documents this kind of regime, becoming suboptimal once its initial error exceeds $2/3$. The analogous condition here is a back-view row of $K$ that puts most of its mass on the front. We expect routing to degrade on such prompts, and we will report the threshold rather than hide it.

This design differs from each existing Janus fix. Debiasing Scores and Prompts of 2D Diffusion for View-consistent Text-to-3D Generation clips over-aggressive scores early in optimization and drops view-contradicting words from the prompt. That suppresses the biased gradient but does not redirect it. Hallo3D asks large multimodal models to detect hallucinations and turns them into negative prompts, which is expensive per iteration. Carve3D fine-tunes a multi-view diffusion model with reinforcement learning, using a reward from reconstruction followed by re-rendering. That changes the generator itself rather than correcting at test time. In-N-Out aligns noise latents across views, but only for forward-facing cameras. Our method explicitly targets full 360-degree orbits, where the frontal-bias channel matters most.

Baselines

The naive reference is standard score distillation with view-dependent prompt suffixes. On countable-face prompts, Debiasing Scores and Prompts reports a face-generation success rate of 29.3% for its view-prompted baseline, SJC. We rerun this baseline in our own pipeline rather than relying on the reported number.

The strongest direct competitor is D-SDS from the same paper. It is also test-time and tuning-free, and it targets exactly the early-iteration frontal bias we model. That paper reports 90.42% user preference for view consistency over SJC, but we found no reported D-SDS face-success rate on Lacuna. We run D-SDS on the same backbone, prompts, seeds, and iteration budget as our method, and give both methods the same small hyperparameter sweep. Beating it means a higher face-success rate and lower A-LPIPS, a multi-view consistency measure used in that paper, at equal CLIP similarity, plus a head-to-head user preference above 50%. We also run routing on top of D-SDS. If the gains add up, the two mechanisms are distinct.

As a compute-heavier reference, we run Hallo3D on its 80-prompt set, where it reports only user-study and CLIP-score results. We also report the Multi-view Reconstruction Consistency metric, which reconstructs a 3D object from the views and scores the re-rendered images with LPIPS. Carve3D reports 0.0606 on this metric on the 415-prompt DreamFusion set, against 0.0892 for Instant3D-10K. Because Carve3D is a fine-tuned multi-view generator, we treat its number as context rather than as a matched comparison.

Three ablations isolate the key ingredient. The first uses an identity kernel, which is plain distillation with pose scoring but no routing. The second uses a uniform kernel, the depolarizing analogue; the theory says this should give almost no gain. The third rejects mismatched views instead of routing them. A fourth uses an oracle kernel measured from ground-truth renders and gives an upper bound.

Why not just…?

Why not just append view prompts or remove contradictory words? Debiasing Scores and Prompts shows view prompts alone reach only 29.3% face success. Hallo3D documents multi-headed failures in several prompted pipelines. Prompting changes $K$ somewhat but leaves it strongly biased toward the front. Our measured kernels will quantify how much bias remains after prompting.

Why not just clip early gradients, as D-SDS does? This is the real threat, because clipping is cheap and already works well. The difference is that clipping discards the misattributed signal, while routing reuses it at the correct camera. If routing does not beat D-SDS, or add to it, on the prompts with the strongest frontal bias, the core claim fails. The matched comparison above settles this.

Why not just reject back-view samples that a pose estimator flags as frontal? Rejection is the pure-state reflex of throwing away "noisy" measurements. It keeps only part of the data and still assumes clean labels for everything it keeps. Our rejection ablation tests directly whether routing matters beyond filtering. If rejection matches routing, we will report the simpler method.

Why not just train on more 3D data or use a multi-view generator? Carve3D found that Instant3D-100K, trained on ten times more supervised 3D data, was still outperformed by a model fine-tuned with only 30 curated prompts. Multi-view generators do weaken the case for 2D-prior distillation. We therefore scope our claim to the regime of 2D priors, where texture quality and prompt coverage remain strongest.

Proposed Experiments

We evaluate on the D-SDS 70-prompt set, reporting countable-face success, A-LPIPS, and CLIP similarity. We also use the 415-prompt DreamFusion set with the reconstruction-consistency metric, and Hallo3D's 80 prompts with CLIP-score variants. Blinded user studies will follow the protocols of the prior work.

Two checks are decisive. First, across prompts, the measured asymmetry of $K$ should predict how badly plain distillation fails, with the correlation computed on held-out prompts. Second, routing gains should concentrate on the prompts with high asymmetry but below the ill-conditioning threshold, and vanish under the uniform-kernel ablation. If failure rates do not track the kernel, the mixed-measurement account of the Janus problem is wrong, whatever the benchmark numbers show.