Object Panoramas: Seam-Free Spherical Canvases for Single-Image 3D

When a model reconstructs a 3D object from one photo, the hardest part is the back, which the camera never saw. Many fast reconstructors write their output onto flat 2D grids, such as texture atlases, triplanes, or six orthographic views. A flat grid has edges, and the back of the object tends to fall on them. This proposal represents the object on a sphere instead. Picture standing at the object's center and looking outward in every direction. Each direction records how far away the surface is and what it looks like. That map is stored as an equirectangular image (ERP), the 2:1 rectangular "world map" layout of a sphere, where horizontal position is longitude around the object and vertical position is latitude. We call this image an object panorama.

The central observation is simple. In an object-centred frame, turning the object about its vertical axis is exactly a sideways circular shift of the object panorama. If we put the input camera at longitude 0, the unseen back of the object lands at longitude 180°. In a planar ERP that longitude is the left and right edges of the image, which are the same line on the sphere. A standard network that treats those edges as a hard boundary must complete the hidden back from two disconnected halves. Each half extends its own adjacent side view, and nothing forces them to agree. We hypothesize that this is one concrete source of the duplicated, mismatched backsides that are the feed-forward cousin of the Janus problem, the "two-faced" artifact in 2D-to-3D generation. We propose a single-pass reconstructor that predicts 3D Gaussians on a gravity-aligned object panorama, processed with wrap-around operations. We also design experiments that test whether seam continuity, rather than simpler fixes, is what removes these artifacts.

Core Claim

Three steps make the unseen hemisphere a single contiguous region, rather than a region split across a boundary. The first is pose normalization reduced to its essentials: predict the object's up direction, and place the input view at a fixed longitude. The second is representing the object as a layered spherical depth-and-appearance map. The third is processing that map with seam-free spherical operations. The claim makes a measurable prediction. Under planar processing, reconstruction error and front–back duplication should concentrate in a longitude band around 180°, and that concentration should disappear with wrap-around processing. It should also reappear if we deliberately move the seam back into the hidden region.

What Prior Work Does and Does Not Do

Three lines of work motivate the idea without implementing it.

The first is panorama work, which has repeatedly shown that the ERP seam and the poles break planar networks. PanoSwin wraps shifted attention windows across the longitude seam and uses split-and-rotate shifts at the poles. It reports 29.4 mAP on seam-crossing detection on 360-Indoor versus 24.0 for a standard Swin transformer. DA² has image patches cross-attend to explicit azimuth and polar coordinate embeddings, and it predicts panoramic depth in a single ~0.1 s pass. PanoDiffusion rotates its latent panorama by 90° at every denoising step so the seam is repeatedly moved into the network's view. All three work on camera-centred scenes: the viewer sits inside a room and looks out. None uses an object-centred sphere, and none outputs explicit 3D primitives. We reuse their seam-handling techniques as components and turn the geometry inside out.

The second line is feed-forward object reconstructors that output flat charts. SF3D regresses a triplane from a single image, extracts a mesh, and applies a fast cube-projection UV unwrap, with no continuity across chart boundaries. CRM first generates six orthographic views plus canonical coordinate maps with diffusion. It then maps them onto three orthogonal planes with a convolutional U-Net, which trades one seam for many cube edges. Dream, Lift, Animate lifts Gaussians into a SMPL-X UV atlas and decodes them with a planar ConvNet. Its authors report color leakage between adjacent parts. These papers constrain the design: canonical parameterizations help, but chart cuts inherited from atlases or cubes create boundary artifacts. Our representation has one wrap-around seam and two poles, and each has a known fix.

The third line is anti-Janus work, which operates on multi-view images rather than on the reconstruction's own representation. Carve3D fine-tunes a multi-view diffusion model with reinforcement learning. Its reward re-renders a reconstruction from a feed-forward reconstruction model and compares it to the generated views, lowering its consistency score (MRC) from 0.0892 to 0.0606 on DreamFusion prompts, at the cost of an expensive loop. CAT3D documents that inconsistent generated views produce blur and floaters once reconstructed. We do not replace these generators. We ask whether a reconstructor whose output space has no seam behind the object avoids manufacturing inconsistency in the first place.

Research Program

The representation is a layered object panorama of size $H \times 2H$ in a frame whose vertical axis is the predicted up direction and whose longitude 0 points at the input camera. Each pixel corresponds to a unit direction $d(\phi,\theta)$ from the object centre, with longitude $\phi$ and latitude $\theta$. A single ray from the centre misses the concave parts of non-convex objects, so each pixel carries $K$ layers; we start with $K=2$ for entry and exit hits and test $K=4$. Each layer $k$ stores a radius $r_k$ and the parameters of one 3D Gaussian: scale, rotation, opacity, and color. That Gaussian is placed at

$$ \mu_k(\phi,\theta) = c + r_k(\phi,\theta)\, d(\phi,\theta), $$

where $c$ is the predicted object centre. The output is renderable with standard Gaussian splatting, as in pixelSplat. Unlike pixelSplat, the primitives come from one global canvas rather than from separate views, so the redundant overlapping Gaussians that paper reports should not arise.

The network has three parts. A DINOv2-style image encoder, as in SF3D, reads the photo. A small head predicts the up vector and centre. A decoder transformer operates on the panorama grid. Its queries carry DA²-style spherical coordinate embeddings and cross-attend to image tokens, and its self-attention uses PanoSwin-style windows that wrap across longitude and bridge the poles. Losses are rendering losses against multi-view renders, plus a direct radial-depth loss from ground-truth meshes. Every per-pixel loss is weighted by $\cos\theta$ so the stretched polar rows do not dominate. Training uses OmniObject3D: 6,000 scans in 190 categories with 100 renders each. The scans are captured upright, which gives up-axis supervision. We augment by random yaw, which is an exact circular shift of the target canvas. Evaluation holds out Google Scanned Objects.

The analysis that matters most is a longitude-resolved error profile. For every method, we measure geometry and appearance error as a function of angular distance from the input view. We also measure a front–back duplication score: how similar the rendered back is to the mirrored rendered front, minus the same quantity on the ground-truth mesh. A positive value means the method invents a second front. For humans, rendered from THuman2.0 scans, we additionally count face detections on rendered back views.

Baselines

The naive reference is the same network, data, and training schedule with an ordinary planar ERP decoder. It uses zero padding, standard 2D positional encoding, and non-wrapping windows. This isolates everything except seam handling. It is also where we expect the error profile to peak at 180°.

The strongest existing single-image method on our benchmark is SF3D. It reports a Chamfer distance of 0.096 on GSO, against 0.111 for TripoSR and 0.195 for LGM (SF3D). CRM reports 0.0094 Chamfer and 16.22 texture PSNR on GSO (CRM). The order-of-magnitude gap from SF3D's figure implies a different normalization protocol, so we will not compare against either number directly. We will run released SF3D and CRM checkpoints under one protocol: our GSO split, a single evaluation script, and identical input views. Because those models were trained on larger data than OmniObject3D, we also train a controlled triplane decoder with our encoder, data, and GPU budget. That separates representation effects from data effects. Beating SF3D means lower Chamfer and a lower duplication score on the rear 90° band under the shared protocol. Matching it on the front band is acceptable.

The generative pipeline baseline takes views produced by CRM's six-view diffusion stage and feeds them to our reconstructor and to the triplane control. This checks whether our gains survive when a multi-view prior already supplies the back.

The key ablations add seam handling in order of cost. First comes circular horizontal padding only, the DreamPBR-style fix. Then come wrapped windows without polar bridging. Then the full model with spherical coordinate attention. Finally, a pose ablation replaces the predicted up vector with the raw camera frame.

Why not just…?

Why not just move the seam to the front, where the input photo constrains everything? This is the most serious threat. It costs nothing: place the input at longitude 180° so the seam falls on observed surface. If this matches the full model, the honest conclusion is that seam placement, not seam-free architecture, is what matters, and we would report that as the finding. The experiment is a direct comparison of "planar, seam at front" against "wrap-around, seam irrelevant". We will also train with random seam placement, the static analogue of PanoDiffusion's rotation. We expect front placement to help but not to fully close the gap, because the top and bottom rows still meet at the poles and yaw augmentation keeps moving content across the edge.

Why not just apply circular padding and stop there? For an object panorama, unlike an irregular UV atlas, circular padding is topologically correct in longitude. That is exactly why this representation is attractive. It may well suffice, and it is the first rung of the ablation ladder. PanoSwin's results suggest that padding leaves the poles broken and does not give attention layers wrap-around proximity. If the error profile flattens at 180° with padding alone, we adopt the cheaper design.

Why not generate multi-view images and reconstruct, as CRM or a Carve3D-tuned generator does? Multi-view diffusion addresses Janus at the image level, and it is complementary to this proposal rather than a rival. Carve3D shows that even dedicated multi-view generators remain inconsistent enough to need an expensive reinforcement-learning loop. CAT3D shows that the inconsistencies turn into blur once reconstructed. The generative-pipeline baseline tests whether a seam-free reconstructor absorbs those inconsistencies better than a triplane one. If it does not, the contribution narrows to the purely feed-forward regime.

Proposed Experiments

The decisive check has two parts. The planar baseline's longitude-resolved error and duplication score must rise measurably in the band around 180° on GSO. The full model must flatten that rise without losing accuracy on the observed front. If the planar baseline shows no rear-band excess, the hypothesis is wrong and the project reduces to a representation study.

Second, the seam-relocation and circular-padding ablations determine how much of any gain comes from cheap fixes.

Third, the human-scan face-count test asks whether reduced duplication appears on the canonical Janus case.

Fourth, the layer ablation ($K=2$ versus $K=4$) on OmniObject3D's most concave categories measures where the spherical-ray assumption breaks. We will report those categories as limitations rather than average them away.

Finally, we report runtime next to SF3D's 0.3–0.5 s per object, since the method's case rests on remaining a single forward pass.