Effective Resistance Predicts Where Multi-View Diffusion Grows Extra Faces

Generators that produce many camera views of one object at once often fail in a characteristic way. The back of a head grows a second face, or a dog gains a third ear. This is the Janus problem. Recent multi-view diffusion models reduce it by letting views exchange information. A diffusion model is a generator that learns by gradually denoising random noise into images. The exchange happens through cross-view attention, where each view looks at features from other views. When there are hundreds of views, letting every view attend to every other becomes expensive, so practical systems wire views together sparsely. A single reference view may feed all the others, or each view may talk only to its neighbors along the camera orbit. Nobody has asked which wiring is right, or whether the wiring itself predicts where the failures appear.

We propose to treat the views as nodes in a noisy consensus network and borrow a precise result from networked control. When agents average with neighbors while being jostled by independent noise, the disagreement between any two agents is set by the effective resistance between them. This is the electrical resistance measured between the two nodes if every edge were a unit resistor. Our claim is that the same quantity predicts cross-view inconsistency in multi-view diffusion. Janus artifacts should concentrate on view pairs with high effective resistance in the attention graph, such as the back view in a star centered on the front view, or the antipode on a ring. Under a fixed attention budget, the graph that minimizes total resistance, not the one with the most geometric overlap, should give the most consistent objects. The same framing yields a testable prediction for real-sensor conditions. Noisy depth and pose corrupt the edges themselves, so the best graph should become sparser and more selective as sensor noise rises.

Core Claim

The analogy rests on first-order consensus under white noise, $dx = -Lx\,dt + \sigma\,dW$. Here $x$ stacks the node states, $L$ is the graph Laplacian (degree matrix minus adjacency), and $\sigma$ is the noise scale. The stationary covariance of deviations from the mean is $(\sigma^2/2)L^{+}$, so

$$\mathrm{Var}(x_i - x_j) = \tfrac{\sigma^2}{2}\,R_{ij}, \qquad R_{ij} = (e_i - e_j)^\top L^{+} (e_i - e_j),$$

where $L^{+}$ is the pseudoinverse of $L$ and $e_i$ is the $i$-th unit vector. Summed over pairs, this becomes the network coherence studied in A Resistance-Distance-Based Approach for Optimal Leader Selection in Noisy Consensus Networks. That paper shows noise-induced variance is proportional to summed resistance distances. On performance of consensus protocols subject to noise: role of hitting times and network structure ties the same degradation to random-walk hitting times. Scale-free Loopy Structure is Resistant to Noise in Consensus Dynamics in Complex Networks proves a lower bound of $1/(2\rho)$, with $\rho$ the average degree. It shows that graphs with loops of diverse lengths approach this bound independent of size, whereas paths and stars do not. These papers supply the method.

We do not claim diffusion denoising is linear consensus. The transition to synchronization of networked systems warns that its Laplacian-based predictions rely on linearization near agreement and degrade in strongly nonlinear regimes. Our claim is therefore empirical and falsifiable. Per-pair inconsistency in generated views should rank-correlate with $R_{ij}$ of the attention graph, across graph families with equal edge counts. Ranking graphs by Kirchhoff index (the sum of all $R_{ij}$) should predict their object-level consistency.

Two mechanisms from the Janus literature motivate the noise model. In-N-Out identifies independent starting noise and independent denoising drift as the drivers of cross-view divergence. DreamBooth3D and Understanding Pure CLIP Guidance for Voxel Grid NeRF Models show that a per-view prior pulls unconstrained views toward canonical, face-forward content. That pull is a per-node bias that coupling must overcome. In-N-Out's fix warps latents from one reference view, which is a star graph. The authors restrict it to forward-facing scenes, which is exactly where a star's resistance to far views stays small.

Research Program

We build a multi-view diffusion sampler whose cross-view attention is restricted by an explicit mask, a graph over the $N$ target cameras. This follows the camera-conditioned design of CAT3D, which generates 80 to 960 views linked by attention. The mask is the only thing we vary. Each view attends to itself and to its graph neighbors.

The first part of the program is measurement. For star, ring, k-nearest-neighbor-in-pose, random sparse, and resistance-optimized graphs, all at matched edge budgets, we compute pairwise inconsistency between generated views. Inconsistency is measured as reprojection error of generated view $i$ warped into view $j$ through the reconstructed geometry. We regress it against $R_{ij}$, against geodesic camera angle, and against both together. Camera angle is the confound that matters, because nearby cameras are both low-resistance and easy to match. We separate the two by including graphs in which angularly distant pairs have low resistance thanks to chords across the orbit.

The second part is design. Given an edge budget $B$, we choose edges to minimize the Kirchhoff index, weighting candidate edges by geometric overlap. This is a known convex relaxation of graph design, with greedy rounding.

The third part brings in the decoupled estimation and control and sim-to-real directions. We borrow a structural idea from networked control. Optimal Controller Synthesis and Dynamic Quantizer Switching for Linear-Quadratic-Gaussian Systems shows that a joint problem splits cleanly when the communication schedule depends only on quantities the controller cannot influence, so the schedule can be solved offline. Linear Quadratic Mean-Field Games with Communication Constraints shows the same split when the scheduler sits at the plant. We apply this as a hypothesis. The attention graph is designed offline from camera poses and a sensor-noise estimate alone, independent of the content being generated. We test whether content-adaptive graphs, rewired mid-sampling from current disagreement, beat it. If they do not, separation holds empirically and graph design becomes a cheap pre-computation.

We also borrow Maity's innovations idea. Neighbors pass the residual between a view's latent and the geometry-warped prediction from its neighbor, rather than raw features. This is intended to keep warping errors from compounding along long paths.

Sensor noise enters because real depth and pose are noisy. Noisy geometry makes each warped edge a source of error, so adding edges both lowers resistance and injects edge noise. The consensus model then predicts an interior optimum in edge count that shifts toward sparser, more reliable edges as depth and pose noise rise. That prediction is new and checkable.

Baselines

The naive reference point is independent per-view generation with no cross-view attention, the regime that produces Janus artifacts in DreamBooth3D. Every other configuration should beat it. The quantity of interest is by how much.

The star baseline reproduces In-N-Out's reference-view coupling, including its explicit initial latent alignment and implicit intermediate alignment through cross-view attention. It is the strongest existing sparse design, and its geometry-grounded warping is a real competitor near the reference. We run it with the same backbone, the same number of sampling steps, and the same edge budget as our graphs.

The dense baseline is full all-to-all attention in the style of CAT3D. It is the quality ceiling at quadratic cost. On the Mip-NeRF 360 3-view task, CAT3D reports PSNR 16.62, against 15.50 for ReconFusion and 14.44 for ZeroNVS. Our backbone will not be CAT3D itself, so we first report our own dense-attention number and treat it, not 16.62, as the within-study ceiling. We count as success recovering at least 90% of the dense model's PSNR gap over independent views while using at most 25% of its attention edges.

The strongest consistency-specific competitor is Carve3D. It fine-tunes a multi-view diffusion model with reinforcement learning, rewarding Multi-view Reconstruction Consistency (MRC), the LPIPS distance between generations and their re-rendering through a fast reconstructor. On the 415 DreamFusion prompts it reports MRC 0.0606 against 0.0892 for the Instant3D-10K base. Our method is training-free and changes only sampling, so the fair comparison applies both to the same base model. We then test whether resistance-optimized graphs add to Carve3D's gain. Carve3D's fine-tuning is expensive, so we report its compute alongside.

Two ablations isolate the key ingredient. A random sparse graph with identical edge count and degree sequence tests whether resistance, rather than degree, matters. The $1/(2\rho)$ bound says degree alone sets a floor, and the loopy-network result says the arrangement of edges decides how close a graph gets to that floor. A k-nearest-neighbor graph in camera-pose space at the same budget tests whether resistance adds anything beyond geometric proximity. The innovations-coupling variant is compared to raw-feature coupling on the same graph.

Why not just…?

Why not just use full attention everywhere? For the tens of views in most object benchmarks, it is affordable and is our ceiling. The question only matters at the hundreds-of-views scale CAT3D targets, or on memory-constrained robot hardware. If dense attention stays affordable at the scales people actually use, the design contribution shrinks to the explanatory one: resistance predicts where failures occur. We report attention FLOPs per configuration so readers can judge the trade.

Why not just connect each camera to its nearest neighbors in pose? This is the real threat, because proximity is what warping needs. But a ring or k-nearest-neighbor graph on an orbit of $N$ views has antipodal resistance growing linearly in $N$. The consensus model predicts front and back will decouple, which is the Janus signature. The matched-budget comparison, reported separately for antipodal pairs, settles it. If k-nearest-neighbor graphs match resistance-optimized graphs on Janus rate, we drop the design claim.

Why not just fine-tune for consistency as Carve3D does? Fine-tuning changes the per-view prior, while graph design changes how disagreement is damped. The two should compose. Carve3D also needs a reconstruction loop per reward evaluation. If Carve3D alone closes the gap and graphs add nothing on top, that is a clean negative result we would report.

Why not just train with noisy depth and pose so the model learns to ignore bad geometry? Randomizing inputs is the standard sim-to-real remedy. It is orthogonal to graph design, and we include a noise-augmented backbone as a condition. The consensus model predicts that even a robust node model benefits from pruning unreliable edges. The noise sweep below tests this directly.

Proposed Experiments

For object generation, we use the 415 DreamFusion prompts with MRC as the primary metric. We add a Janus rate: the fraction of objects with duplicated faces or limbs, scored blind by human raters on a fixed subset. For scenes, we use the Mip-NeRF 360 3-view task with PSNR, SSIM, and LPIPS.

For the sim-to-real noise axis, we use TartanAir, which provides ground-truth depth and camera poses. We condition warping on geometry corrupted by controlled depth and pose noise, sweeping from clean to levels that mimic monocular estimates. TartanAir's own ORB-SLAM results illustrate how large realistic errors get: absolute trajectory error rises from 0.165 by day to 9.385 by night in Abandoned-factory. We also run on real Mip-NeRF 360 captures with poses estimated from images, as a real-sensor check.

The first decisive check is the pairwise regression. If effective resistance does not explain inconsistency beyond camera angle, the central claim fails. The second is the matched-budget ranking. The Kirchhoff index should order graph families by MRC and Janus rate, with the resistance-optimized graph beating random and k-nearest-neighbor graphs at equal degree, especially on antipodal pairs. The third is the noise sweep. The consistency-optimal edge budget should shrink as geometric noise grows, which is the edge-noise prediction that distinguishes our model from "more coupling is always better." The fourth is the separation test. Content-adaptive rewiring should fail to beat the offline pose-and-noise-only graph by more than seed variance. Each check can fail independently, and the note's claims are scoped so that each failure removes a specific one.