Feed-Forward Curve Primitives for Thin Structures

Fast 3D reconstruction networks now turn a handful of photos into a renderable 3D scene in well under a second. They do this without any per-scene training loop, and the result renders in milliseconds. They also share a blind spot. Wires, chair legs, bicycle spokes, fork tines and railings come out blurred, broken into fragments, or missing. These networks attach one small 3D blob (a "Gaussian", a fuzzy ellipsoid with color and opacity) to every input pixel at a predicted depth. A wire thinner than a pixel has no single depth at that pixel: the pixel's color mixes wire and background. The network therefore either averages the two depths, producing a ghostly halo, or drops the wire. We propose a network that, in the same single forward pass, also predicts a small set of 3D curves. Each curve is a smooth parametric line with a continuous width, rendered as a chain of thin Gaussians. Curves are placed by matching thin image features across views, not by assigning per-pixel depth.

The two-sentence version is this. Thin structures fail in feed-forward reconstruction partly because the training loss ignores them and partly because "one primitive per pixel at one depth" cannot represent something narrower than a pixel. We add a curve branch that represents such structures natively, and we test whether it beats simply reweighting the loss on an existing model. The work sits where four concerns meet. Thin-structure recovery is the target. Removing per-scene optimization is a hard constraint. Rendering latency is a constraint and a side benefit, because curves are compact. Generalization across object categories is the evaluation regime, since thin parts appear in nearly every category and are rarely the focus of training data.

Core Claim

The claim is that feed-forward prediction of hybrid primitive sets recovers thin structures on unseen categories far better than any per-pixel feed-forward model, even one trained with thin-aware losses. The hybrid set combines pixel-aligned surface primitives for bulky regions with explicit 3D curves for structures a few pixels wide or less. It does this while keeping reconstruction under a second and rendering real-time, with fewer primitives.

What is new is the feed-forward curve branch, and in particular how it locates curves in 3D. In each view, a thin 2D curve together with its camera center defines a plane in space. The 3D curve lies where these back-projected planes from several views intersect. This uses classical line triangulation as an inductive bias and sidesteps the mixed-pixel depth ambiguity entirely.

The second half of the claim is falsifiable and is what makes this worth doing. If an existing feed-forward model trained with the same thin-weighted loss and the same thin-aware allocation of primitives matches our thin-structure recall, then the representation is not the bottleneck. In that case we would report the loss result and drop the curve claim.

Where This Sits

Three lines of feed-forward work motivate the gap without closing it. They are pixelSplat, MVSplat and GS-LRM.

  • pixelSplat predicts per-pixel Gaussians from image pairs with an epipolar transformer.
  • MVSplat builds a plane-sweep cost volume, links opacity to matching confidence, and reconstructs in 44 ms with 12M parameters.
  • GS-LRM regresses per-pixel Gaussians from 2–4 posed views with a large transformer in about 0.23 s.

All three tie each primitive to a pixel and a depth, so a sub-pixel wire has nowhere to go.

Two newer models are the strongest competitors. DepthSplat fuses that cost volume with a monocular depth network, gaining about 2 dB over MVSplat on DL3DV. VisTa3D constrains this design: it benchmarks eleven vision methods on 70 thin objects and finds that monocular depth and multi-view stereo erase thin structures, with an AbsRel of 0.274 for the vision-only baseline on thin objects. A monocular prior is therefore likely to hurt exactly where we care.

SurfSplat replaces ellipsoids with oriented flat discs (2D surfels) whose orientation comes from depth-map gradients. It also exposes a "resolution trap": rendering at 2–4× resolution reveals holes and floaters. Its discs assume a locally two-dimensional surface, which is the wrong shape for a one-dimensional wire. We keep surfels for bulk and add curves for filaments.

ReSplat supplies a method we reuse. Its recurrent network feeds back 2D rendering-error maps over four steps without test-time gradients, beats DepthSplat by 3.5 dB on DL3DV, and uses 246K Gaussians against MVSplat's 3.9M. Its initialization is 16× subsampled and its feedback is photometric, and both choices are hostile to thin structures. We change the feedback signal to edge and coverage error.

Per-scene methods supply the curve machinery and the ceiling.

- Curve-Aware Gaussian Splatting (CurveGaussian) binds Gaussians analytically to Bézier curves and optimizes them against multi-view edge maps, cutting curve counts by 70.5% over two-stage pipelines. It is entirely per-scene. We borrow its curve-to-Gaussian parameterization and amortize it into a network.

  • CurveFusion fuses 1D skeletons from RGB-D video down to 2 mm wires. It shows that curve primitives are the right unit for wires, but it cannot represent surfaces and needs sequential tracking. Our hybrid set addresses exactly that limitation.
  • Curriculum DeepSDF and CrossSDF both show that implicit surface functions forget thin parts under uniform sampling. Curriculum DeepSDF uses a tolerance curriculum and sign-flip weighting; CrossSDF uses contour-adaptive sampling and a symmetric-difference loss. Together they motivate our thin-aware allocation and loss, and they define the reweighting baseline we must beat.

On the data side, OmniObject3D documents failures on fork tines and concavities across 190 real categories. That makes it the natural zero-shot test of cross-category generalization.

Research Program

The bulk branch. We start from a DepthSplat-style backbone: a multi-view cost volume plus monocular features, decoding per-pixel surfels in the manner of SurfSplat.

The thin head. A lightweight head predicts four quantities per pixel:

  • a thin-structure probability;
  • a 2D tangent direction;
  • a coverage fraction, meaning what share of the pixel the structure occupies, which encodes sub-pixel width;
  • an embedding used to group pixels into curve instances.

Suppressing halos. Where thin probability is high, the bulk branch's opacity is multiplicatively suppressed. This prevents the averaged-depth halos that per-pixel models produce on wires.

Proposing curves. Grouped thin pixels form per-view 2D curve proposals. Each proposal becomes a token that cross-attends to thin features in other views, along epipolar bands. A decoder then regresses cubic Bézier control points and a radius profile.

Locating curves in 3D. For curves with good baseline, the decoder is trained to agree with an analytic estimate: the intersection of back-projected planes. Writing $\pi_v$ for the plane through camera $v$'s center and the view-$v$ 2D curve segment, a 3D segment point $X$ should satisfy $$ \sum_v w_v \, d(X, \pi_v)^2 \to \min, $$ where $d$ is point-to-plane distance and $w_v$ is the network's per-view matching confidence. The confidence matters because this estimate is degenerate when a curve runs nearly parallel to the epipolar lines. In that case $w_v$ is learned to fall back on the cost volume.

Rendering. Curves are converted to chains of elongated Gaussians, following CurveGaussian. They are sorted and alpha-blended in the same rasterizer as the surfels, so rendering speed is unchanged in kind.

Refinement. Two to four ReSplat-style recurrent steps then update both primitive sets. The feedback here is a stack of two error maps:

  • the photometric error;
  • the difference between rendered and predicted thin-coverage maps.

The coverage error is what keeps the refinement from treating wires as noise.

Training. Training uses multi-category synthetic renders from Objaverse with category tags. Meshes give ground-truth depth, thin masks (pixels where the projected local thickness is below $k$ pixels, computed from the mesh's medial thickness), and skeleton curves for auxiliary supervision. Training then continues on DL3DV and RealEstate10K with rendering losses only.

The training loss adds a thin-weighted term to standard photometric and perceptual losses: $$ \mathcal{L} = \mathcal{L}_{\text{photo}} + \lambda \, \frac{\sum_p m_p \lVert \hat I_p - I_p \rVert_1}{\sum_p m_p} + \mathcal{L}_{\text{cov}}, $$ where $m_p$ is the thin mask at pixel $p$, $\hat I_p$ and $I_p$ are rendered and true colors, $\lambda$ balances the term, and $\mathcal{L}_{\text{cov}}$ is the binary cross-entropy on coverage. Normalizing by the mask area keeps a few hundred wire pixels from being drowned out by a million background pixels. This is exactly the failure that Curriculum DeepSDF and CrossSDF address with reweighting.

Baselines

Naive reference. The do-nothing reference is DepthSplat as released, fine-tuned on our training mix with its original loss. It shows how badly a strong per-pixel model does on thin regions when nobody asks it to care.

Strongest existing feed-forward competitors. These are ReSplat and SurfSplat, retrained on the identical data mix with the same input views, resolution and GPU-hour budget.

  • ReSplat is the most accurate feed-forward model on DL3DV in the file, reporting a 3.5 dB PSNR gain over DepthSplat on 8-view DL3DV.
  • SurfSplat reports 27.537 PSNR on RealEstate10K and 24.897 on its 1024×1024 high-resolution metric. It is the strongest on geometric continuity.
  • MVSplat's reported 26.39 PSNR on RealEstate10K anchors the global-quality scale.

On thin-structure benchmarks there is no reported result found on Lacuna for any feed-forward splatting model, on either VisTa3D or OmniObject3D thin parts. We will measure all of them ourselves. Beating these competitors means a large gain in thin-skeleton recall and thin-masked PSNR on held-out categories, with global PSNR within a few tenths of a dB.

Key ablation baseline. The most important comparison is DepthSplat-plus-thin. This is the same backbone with our thin-weighted loss, the coverage head, the bulk suppression, and thin-aware primitive allocation, which densifies per-pixel Gaussians on high-thin-probability pixels at full resolution. It gets everything except curves and is trained under the same budget. It isolates the representation from the loss. We count as winning only if our model beats it on thin recall by a margin larger than the seed-to-seed spread across three training runs. If we do not, we report that loss and allocation suffice.

Per-scene references. Per-scene CurveGaussian, and standard 3D Gaussian splatting run for a full optimization, serve as quality ceilings and latency floors. VisTa3D reports that 3D Gaussian splatting struggles on thin objects, so the second may not be a true ceiling.

Ablations. We remove the curves, remove the bulk suppression, swap coverage feedback for photometric-only feedback, and replace plane-intersection lifting with per-pixel depth for curve points.

Why not just…?

Why not just train an existing feed-forward model at higher resolution? GS-LRM is limited to around 512×904. ZPressor reports that DepthSplat and MVSplat run out of memory on an 80 GB GPU beyond 16 views at standard resolution. SurfSplat's resolution trap shows that per-pixel models degrade rather than sharpen when rendered at higher resolution. More pixels give a wire more coverage but still one depth per pixel, so the mixed-pixel problem shrinks without disappearing. We will still test it directly: the bulk-only model at 1.5× input resolution, at matched memory, is one row in the thin-recall comparison.

Why not just initialize with a fast model and run a few hundred optimization steps? This brings back seconds-to-minutes of per-scene cost, and the photometric objective it would optimize is the one that smooths wires away. VisTa3D finds fully optimized 3D Gaussian splatting fails on thin objects. We include DepthSplat followed by 500 steps of optimization as a baseline and report its thin recall alongside its wall-clock time. The question this answers is whether a short optimization budget closes the gap we claim the curve branch closes.

Why not just reweight the loss and allocate more primitives to thin pixels in an existing model? This is the real threat. Curriculum DeepSDF and CrossSDF both obtained large thin-structure gains from weighting and sampling alone, without new primitives. The DepthSplat-plus-thin baseline exists to settle it. Our argument that it falls short is mechanistic: a per-pixel primitive at a mixed pixel must still choose one depth, and nothing ties a wire's estimates together across views. The prediction follows from that argument. The gap between the two should grow as structures get thinner, so we report recall binned by projected width (under 1, 1–3, and 3–8 pixels). If the bins show no gap, the curve branch is not justified.

Why not just segment thin structures in 2D and reconstruct them with a separate per-scene curve method? CurveGaussian documents error accumulation in two-stage edge-then-fit pipelines. Thin Structures in Image Based Rendering reports camouflage failures and 5–20 minutes of processing per scene. Our model learns the thin head jointly with multi-view matching, so evidence from views where the wire contrasts well can compensate for views where it does not. A camouflage split, which we define below, tests this.

Proposed Experiments

Cross-category generalization. We hold out Objaverse category tags that are rich in thin parts from training and evaluate zero-shot on three sets:

  • those held-out renders;
  • GSO;
  • OmniObject3D real scans across its 190 categories, using posed multi-view captures.

For thin-structure benchmarks we use VisTa3D's real scenes with vision-only input. We render depth at the input views and report the benchmark's thin-object AbsRel and A1. The reference points are the vision-only baseline's 0.274 AbsRel and, separately, the touch-assisted 0.205, which our method does not use. Matching the touch-assisted number from vision alone would be a strong result.

Thin-structure metrics. On all sets with meshes we report:

  • thin-skeleton recall and precision, meaning the fraction of ground-truth skeleton within a tolerance of the reconstruction, binned by projected width;
  • junction recall, in the spirit of CurveFusion's evaluation;
  • PSNR and LPIPS restricted to thin masks;
  • Chamfer distance on thin parts only.

Global quality. On RealEstate10K and DL3DV we report global PSNR, SSIM and LPIPS, plus SurfSplat's high-resolution rendering protocol, to show that bulk quality does not regress.

Latency and cost. We report reconstruction time, rendering frames per second at 1080p, and total primitive count.

Camouflage split. A camouflage split, built by selecting test views where the thin mask's local color contrast with its surroundings falls below a threshold, probes multi-view compensation.

Decisive checks. The project succeeds only if all three of the following hold.

  • On held-out categories, our model beats DepthSplat-plus-thin on sub-3-pixel skeleton recall by more than the seed spread.
  • Global DL3DV PSNR stays within a few tenths of a dB of retrained ReSplat.
  • Reconstruction stays under a second and rendering stays real-time with fewer primitives than the bulk-only model.

A positive result in the narrowest width bin, together with the plane-intersection ablation losing accuracy there, would show that geometry-aware curve placement, not reweighting, is what recovers the thinnest structures.