Systems that turn text into 3D objects usually work by repeatedly rendering a candidate object from random cameras and asking a 2D image generator how each rendering should change. That generator is a diffusion model, which learns to turn random noise into images step by step. Two failures recur. Objects grow extra faces on their backs, which is called the Janus problem. Highlights and shadows also get painted permanently into the surface color, which is called baked-in lighting, so the asset looks wrong when moved under new lights. This note argues that both failures are the same bug that signal processing solved decades ago for analog-to-digital converters. The bug is deterministic quantization error that is correlated with the signal. The fix is dithering: add a small known random offset before a coarse decision so that, on average, the decision becomes an unbiased and smooth function of the input.
The concrete observation is that standard score distillation sorts two continuous nuisance variables into coarse bins. Camera azimuth is turned into a hard "front / side / back view" text label. Lighting is held fixed or drawn from a few setups. The image prior's answer is then snapped to whatever its training images look like for that bin, which is usually a canonical, front-facing, studio-lit object. Each rendering is out of distribution for the prior, so the prior projects it onto its image manifold. Because that projection error is the same every time a given view or light comes up, averaging over thousands of optimization steps never cancels it, and the error accumulates into duplicated faces and highlights painted into the color map. We propose dithering these nuisance variables with known random offsets, at an amplitude set by the bin width. Every query to the prior then stays an ordinary, in-distribution prompt and image, while the guidance becomes unbiased in expectation.
Core Claim
The idea is borrowed from one-bit compressed sensing. With a hard threshold, $\text{sign}(z)$ throws away the signal's scale, and simple decoders hit an error floor that more measurements cannot remove. The Generalized Lasso for Sub-gaussian Observations with Dithered Quantization. shows that Projected Back Projection stalls at such a floor, while dithered measurements give error that keeps decaying, roughly as $\sqrt{\log m / m}$ in the number of measurements $m$. Thrampoulidis et al. explain why: the rate at which a random threshold is crossed encodes the magnitude that a fixed threshold erases. We reuse this mechanism. We do not reuse the recovery theorems.
Applied to view labels, let $\phi$ be the camera azimuth, $Q_\Delta$ the rule that rounds $\phi$ to the center of its label bin of width $\Delta$, and $\tau$ a known offset drawn uniformly from $[-\Delta/2, \Delta/2]$. The standard dithering identity then reads
$$\mathbb{E}_\tau\left[Q_\Delta(\phi + \tau)\right] = \phi .$$
In words, the label the prior receives is always one of its familiar words, but how often each word appears varies smoothly and without bias with the true angle. A side view that sits just inside the "front" bin no longer gets told "front" every single time.
For lighting, the dither is a known environment light sampled from a broad family of real lighting conditions. A physically based renderer renders the material under that light, which is the 3D analogue of subtracting the dither after the decision. The prediction from quantization theory is as follows. The prior's preferred lighting creates a view- and position-dependent pull on albedo, meaning the surface's base color. Under dithering, that pull becomes independent of where the surface point is, so it reduces to a near-global offset rather than structured highlights and shadows. The theory also predicts its own failure modes. Dither narrower than the bin, or narrower than the spread of the prior's lighting bias, should leave a residual floor. Much wider dither should add variance and slow convergence. The result should be a U-shaped curve in dither amplitude with a predictable minimum. Random lighting augmentation that is not matched in amplitude makes no such prediction.
There is also a design rule that separates dithering from smoothing, and it comes directly from the out-of-distribution literature. One could instead interpolate the "front" and "side" text embeddings, or average lighting into a flat ambient term. Both produce queries the prior never saw in training. Algorithmic Aspects of Inverse Problems Using Generative Models and Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems show that such off-manifold inputs are exactly what generative priors answer with forced projection and hallucinated content. Dithering keeps each individual query on the prior's manifold and achieves smoothness only in expectation.
Relation to Prior Work
Several papers motivate the diagnosis. DreamBooth3D identifies the prior's strong bias toward front-facing views, which is the systematic error our dither targets. In-N-Out shows that independent noise seeds push views down divergent paths, which is a variance problem rather than our bias problem. We therefore keep seeds as a controlled factor. CAT3D describes how conflicting per-view signals average into floaters and blur, and the gap analysis in the research file argues that this geometric inconsistency leaks residual error into albedo. That argument is why we treat Janus and baked lighting together rather than as separate projects.
Three lines of work attack these failures with heavier machinery. Carve3D fine-tunes the prior with reinforcement learning against a reconstruction-consistency reward. CasTex switches to explicit physically based texture maps and pixel-space distillation, yet still reports baked lighting. ExMesh++ and TensoIR remove baked shadows by physical decomposition, but they rely on real, consistent multi-view captures. Our proposal changes only the sampling distribution of the queries sent to an unmodified prior. It is therefore complementary to all three and can be stacked on CasTex's pipeline.
Some work constrains the proposal. Generative compressed sensing theory, in A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing and Robust One-Bit Recovery via ReLU Generative Networks, assumes the target lies exactly on the generator's range and uses ReLU networks. Neither assumption holds here, so we claim no recovery guarantee and treat the theory as a source of testable predictions. The Generalized Lasso's sensitivity to the norm bound $R$ similarly warns that a wrongly sized dither will hurt. That warning is why dither amplitude is calibrated, not guessed.
Research Program
We would first build a calibration probe that measures the prior's bias on each nuisance variable. For view, we render reference meshes with clean texture maps from Objaverse at dense azimuths. We then record how strongly the prior's distillation gradient pushes toward adding frontal features as a function of azimuth and label, which gives the effective bias width around each bin edge. For lighting, we render objects with known constant albedo under controlled lights. We record the albedo gradient the prior induces, which gives the spread of its preferred illumination. These two curves play the role of the dither distribution's scale and set the amplitudes.
Second, we implement dithered score distillation on top of CasTex's explicit texture-map pipeline and a standard text-to-3D geometry stage. Each step draws a known azimuth-label dither and a known environment light. The renderer uses that same light, so shading is attributed to the light and not to the albedo. The prior and all other settings are unchanged.
Third, we test the specific predictions. Undithered optimization should show an error floor that persists as steps increase, while the dithered version keeps improving. Error should be U-shaped in dither amplitude, with the minimum near the calibrated width. Dithering should beat interpolated smoothing that matches it in expectation, because each dithered query stays in distribution.
Baselines
The naive reference is plain score distillation with hard azimuth-label bins and the fixed lighting setup used by CasTex. It runs on the same prior, geometry initialization, step count and seeds. Any gain over it measures what dithering adds, and this is the baseline in which we expect the error floor to be visible.
The strongest existing texture method is CasTex, run with its released settings on its 410-mesh Objaverse subset at its reported cost of about 12 minutes per mesh for two stages. Our dithered variant gets the same wall-clock budget. CasTex is the right competitor because it already uses physically based maps and pixel-space distillation and still admits baked lighting. A remaining gap would therefore show that parameterization alone is insufficient. The file does not give CasTex's FID or KID values, so we rerun it rather than quote them. Beating it means lower baked-lighting error at equal or better FID and KID.
For multi-view consistency, Carve3D reports a Multi-view Reconstruction Consistency (MRC) score, a reconstruction-based perceptual error where lower is better, of 0.0606 on the DreamFusion 415-prompt set. That compares with 0.0892 for its Instant3D-10K base, and 68.33% of participants in its human study preferred it. Carve3D is a fine-tuned multi-view generator rather than a distillation optimizer, so it is a reference point and not a like-for-like opponent. We apply MRC to multi-view images rendered from our optimized assets with the same reconstruction model. Reaching Carve3D's level without any fine-tuning would count as a strong result. For relighting quality in the regime where real captures exist, ExMesh++ reports 34.19 dB relighting PSNR on Synthetic4Relight, against 31.00 dB for RelightGS. We use this as an upper reference that a capture-free generative method should approach but is not expected to exceed.
The key ablations isolate the ingredient. The first is random lighting augmentation with an arbitrary, uncalibrated spread. The second is soft interpolated view embeddings with flat averaged lighting, matched in expectation but off-manifold. The third is dither in only one nuisance variable at a time, which tests whether removing Janus also reduces baked albedo as the coupling hypothesis predicts.
Why not just…?
Why not just randomize the lighting during distillation, as many pipelines already do? This is the real threat, and we treat it as such. If uncalibrated random lighting matches calibrated dithering on every metric, the contribution shrinks to an explanation plus the view-label dither. We would say so. The amplitude sweep settles the question. Quantization theory predicts a residual floor below the calibrated width and slower convergence above it. A flat curve would refute the claim that amplitude matters.
Why not just interpolate the view-label embeddings so the conditioning varies smoothly? Interpolated embeddings and flat ambient lighting are inputs the prior never saw. Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems shows that whole-image priors hallucinate training-set structure when given mismatched inputs, which predicts that interpolation will trade bin-edge errors for new hallucinations. The expectation-matched ablation tests this directly.
Why not just use a prior fine-tuned for 3D consistency, such as Carve3D, or fine-tune on 3D asset datasets? Carve3D reports that supervised fine-tuning on 3D data costs texture fidelity, and its reward loop is expensive and prone to reward hacking without strong regularization. Dithering requires no training and can be layered on any prior, including a Carve3D-tuned one. We run that combination to check whether the gains add up or overlap.
Why not just freeze geometry and then decompose materials, as ExMesh++ does? Staged decomposition works when the input views are genuinely consistent captures. Here the supervision comes from a biased prior, so freezing a Janus geometry locks in the defect. We include a staged variant without dither to measure how much baked albedo it leaves.
Proposed Experiments and Decisive Checks
Janus is evaluated on the DreamFusion 415-prompt set. We report MRC on renders, the rate at which a frontal-feature detector fires at back azimuths, and a blinded human preference study.
Baked lighting is evaluated two ways. The first is the CasTex Objaverse subset, using FID and KID together with relighting under held-out environment maps. The second is a controlled probe set of prompts whose correct albedo is known to be uniform, such as "a matte white plaster bust." On these prompts, any spatial variance in albedo, and any correlation between albedo and ambient occlusion, is baked lighting by construction. Synthetic4Relight in a sparse-view setting, where the prior acts as a regularizer, provides a relighting PSNR check against real ground truth.
The decisive checks are the three predictions from the Research Program: an error floor without dithering that disappears with it as steps grow, a minimum in the amplitude curve near the calibrated width, and dithering beating expectation-matched interpolation. If all three hold, Janus and baked lighting are confirmed as quantization bias with a simple, general remedy. If only the lighting result holds, that result alone answers the open baked-lighting problem CasTex reports.
References
- The Generalized Lasso for Sub-gaussian Observations with Dithered Quantization.
- The Generalized Lasso for Sub-Gaussian Measurements With Dithered Quantization
- Algorithmic Aspects of Inverse Problems Using Generative Models
- Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems
- DreamBooth3D: Subject-Driven Text-to-3D Generation
- In-N-Out: Lifting 2D Diffusion Prior for 3D Object Removal via Tuning-Free Latents Alignment
- CAT3D: Create Anything in 3D with Multi-View Diffusion Models
- Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuning
- CasTex: Cascaded Text-to-Texture Synthesis via Explicit Texture Maps and Physically-Based Shading
- ExMesh++: From Multi-View Images to Relightable UV-PBR Mesh Assets via Topology-Adaptive Reconstruction and Decomposition
- TensoIR: Tensorial Inverse Rendering
- A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing
- Robust One-Bit Recovery via ReLU Generative Networks: Near-Optimal Statistical Rate and Global Landscape Analysis