Feed-forward 3D reconstruction trains a network once on many scenes so that it can turn a handful of photos into a full 3D model in a single pass, with no per-scene optimization. The recent models output 3D Gaussian splats, which are small colored, semi-transparent blobs that a renderer can rasterize in real time. They face a choice about where those blobs live. If the network predicts one blob per input pixel, overlapping views produce duplicated, ghosted geometry. If it first pools all views into a 3D voxel grid (a grid of small cubes), it removes the duplicates. But the grid acts as a blur: a chair leg, a cable, or a railing thinner than a voxel is averaged away before any Gaussian is predicted. The reverse is not true of the input images. A thin object that the grid erased is still visible at full resolution in the photos.
We propose to use that fact directly. A coarse sparse voxel grid handles cross-view consistency and removes duplicates. A small learned module then predicts where this coarse model is geometrically wrong. At a fixed budget of continuous 3D points in those regions, the network goes back to the full-resolution image features of every view that can see each point, fuses them with attention that takes visibility into account, and emits extra Gaussians at sub-voxel positions. Training includes geometric supervision at continuous laser-scanned surface points, so the network cannot hide missing geometry behind texture. The claim is that most of the detail lost to voxelization can be recovered at a small, controlled cost in primitives. The recovery comes from re-reading the 2D modality rather than from refining the 3D grid.
Core Claim
At matched primitive count and GPU memory, a voxel-aligned feed-forward splatting model plus a budget of error-targeted sub-voxel queries will recover substantially more thin-structure geometry than either a finer voxel grid or a pixel-aligned model with merging. These queries read full-resolution image features with visibility-weighted cross-view attention. The gain should come specifically from the image re-reading. The same queries fed only interpolated voxel features should recover little, because the voxel features have already lost the information. The gain should also be largest under high-resolution rendering and geometric metrics, and small at the standard 256-pixel evaluation, because low-resolution PSNR masks missing geometry.
Relation to Prior Work
Two papers motivate the gap. VolSplat unprojects multi-view features into a sparse 3D U-Net and regresses Gaussians per occupied voxel with adaptive density. This removes pixel-aligned duplication, but its features are quantized before Gaussian prediction, so its adaptive density can only redistribute information the grid already kept. SurfSplat predicts per-pixel 2D surfels with surface-continuity priors and introduced the high-resolution rendering consistency (HRRC) protocol, which renders at up to 1024×1024. Under HRRC, pixel-aligned models that score well at low resolution expose holes. Its surfels stay one per pixel, so it inherits redundancy on flat regions. We borrow its evaluation and keep VolSplat's grid, but add a route by which sub-voxel detail can re-enter.
Three papers supply the method. FineRecon shows two things: continuous 3D query points that sample high-resolution 2D features recover sub-voxel detail, and supervising at true surface points rather than interpolated grid values gives about a 10% accuracy gain. It queries densely, outputs a signed-distance volume rather than Gaussians, and pays cubic memory. We use the same two ingredients, but only at selected points and with Gaussian outputs. SGFormer lets 3D points cross-attend to 2D feature tokens instead of reading a single projected pixel. Its reported 2–10% robustness gains under sensor stress are our reason for using attention rather than hard projection, because hard projection lets a point pick up features from whatever occludes it. PointFusion and MVX-Net established that fusing image features before 3D pooling preserves thin objects. That is the same principle we apply, in the setting of feed-forward reconstruction.
Two papers are the closest competitors and constrain the design. UniSplat already decodes a voxel scaffold through a point branch for sharp detail and a voxel branch for filling holes. Its point branch, however, anchors Gaussians to all initial geometry points. It does not select points by where the coarse model fails, and it does not re-read full-resolution images at sub-voxel positions. It targets driving scenes. SPLATT3RFUSION goes in the opposite direction, from pixels to fewer primitives. It merges per-pixel points within octree cells by feature similarity, reaching about 15.5% of Splatt3R's primitives on ScanNet++. Its merging is governed by a hand-set threshold, and it can merge nearby thin structures whose features look alike. The pruning literature makes the same point: Text-guided Sparse Voxel Pruning and Text-based Localization of Small Objects both show that aggressive spatial reduction erases thin and small objects irreversibly. We do not prune a dense set. We start sparse and add primitives only where the coarse model's geometric error is predicted to be high.
Research Program
The base model is a VolSplat-style pipeline with calibrated input views. It has an image encoder, depth-guided unprojection into a sparse voxel grid at a deliberately coarse size, a sparse 3D U-Net, and per-voxel Gaussian heads. We add two components.
The first is an error predictor. It is a small head on the voxel features that outputs a score $s_v$ per occupied voxel $v$. The score estimates how far the coarse Gaussians in that voxel are from the true surface. For training, the target is the measured error: the mean distance from laser-scan surface points inside $v$ to the nearest coarse-model surface, rendered as depth. At test time we keep the top-$K$ voxels by $s_v$, where $K$ is a fixed budget expressed as a fraction of occupied voxels. Inside each kept voxel we spawn $M$ continuous query points. Their offsets are proposed from the voxel feature and refined by sampling along the depth ray.
The second component is a visibility-weighted cross-view reader. Each query point $x$ is projected into every source view $i$. It gathers a small window of full-resolution encoder features around its projection, $F_i(\pi_i(x))$, and attends across views with weights
$$w_i(x) \propto \exp\big(q(x)^\top k_i(x)\big)\cdot \sigma\big(\gamma\,(d_i(\pi_i(x)) - z_i(x))\big),$$
Here $q$ and $k_i$ are the learned query and key, $z_i(x)$ is the depth of $x$ in view $i$, and $d_i$ is the coarse model's rendered depth in that view. The term $\sigma(\cdot)$ is a soft visibility gate with sharpness $\gamma$ that suppresses views in which $x$ is occluded. The fused feature is decoded into a small Gaussian, or a 2D surfel following SurfSplat, at the refined sub-voxel position.
Training combines the standard photometric loss at the native resolution with three further terms. The first is a photometric loss on random high-resolution crops. The second is a resolution-agnostic geometric loss in the style of FineRecon: rendered depth is compared against laser-scan points directly, never interpolated onto the grid. The third is the regression loss for the error predictor. Data will be ScanNet++ (ScanNet++), whose sub-millimeter laser scans make thin-structure geometry measurable, together with RealEstate10K for photometric breadth. Every learned baseline is retrained on this same mixture.
We also need a way to score thin structures. On ScanNet++ we will label laser-scan points whose local thickness is below a set width, using a shape-diameter estimate on the mesh. We report recall of these points within a tolerance, stratified by thickness relative to voxel size. To our knowledge from the file this metric has not been reported. No reported result was found on Lacuna for thin-structure recall in any feed-forward splatting model.
Baselines
The naive reference point is VolSplat at our coarse voxel size with no added queries. This is the model whose losses we claim to repair. It is trained on the same data, schedule and GPU count as the proposal. Any improvement must exceed seed-to-seed variance, which we estimate from three seeds.
The strongest existing method on standard metrics is VolSplat, run at its published voxel size. It reports PSNR 31.30, SSIM 0.941 and LPIPS 0.075 on RealEstate10K with 6 input views, at 0.575 s latency and 4.65 GB of memory. We will reproduce this number before retraining on our data mixture. Beating it means matching its RealEstate10K PSNR within 0.2 dB while improving HRRC PSNR and ScanNet++ thin-structure recall, at no more than its memory and with primitive count within 10%.
The second strong competitor is a finer-grid VolSplat. We halve the voxel size and give it the same total memory budget as our model by restricting the scene extent or batch size. This tests whether simple refinement does the same job.
The pixel-aligned side is represented by SPLATT3RFUSION and SurfSplat. SPLATT3RFUSION is swept over its merging threshold to trace a curve of primitive count against quality; the file gives its primitive ratio but no PSNR on ScanNet++. SurfSplat reports HRRC PSNR 24.897 at 1024×1024 on RealEstate10K, which is the number to beat under HRRC.
We also build a dense-query baseline in the style of UniSplat's point branch. It keeps queries at all depth-unprojected points rather than error-selected ones and is decoded by the same reader. This bounds what selection buys at a given primitive budget.
Two ablations isolate the key ingredient. In the first, queries are placed at random occupied voxels, testing whether error-driven allocation matters. In the second, queries are fed trilinearly interpolated voxel features instead of full-resolution image features, testing whether re-reading the images matters. A third ablation replaces the visibility-gated attention with hard single-pixel projection. For geometry we also report against FineRecon, which reports ScanNet F1 75.5 and Chamfer 5.18. We fuse our rendered depths into a mesh under its protocol, and we note that this comparison pits a splatting model against a model built only for surfaces.
Why not just…?
Why not just use a finer voxel grid? Grid memory grows cubically with resolution, as the Direction 1 literature and FineRecon's own scaling limits make clear. Most of the added cells would sit on flat walls that never needed them. Even so, this is the most direct threat, which is why the finer-grid baseline runs at matched memory. If it matches our thin-structure recall at equal memory and primitive count, the proposal's central claim fails and we will say so.
Why not just add FineRecon's resolution-agnostic geometric loss to VolSplat? This is a real competitor because it costs almost nothing. Our expectation is that the loss alone can sharpen what the grid kept but cannot restore what quantization removed, since the voxel features no longer contain it. The interpolated-voxel-feature ablation, which keeps the loss and drops the image re-reading, tests exactly this. If that ablation closes most of the gap, the contribution reduces to a training objective and we would report it that way.
Why not just go pixel-aligned and merge, as SPLATT3RFUSION does? Merging by feature similarity is designed to collapse look-alike neighbors, and thin repeated structures such as railings and chair legs are exactly the look-alike neighbors. Its threshold is also fixed by hand rather than driven by geometric error. The threshold sweep in our experiments shows directly whether any setting reaches our thin-structure recall at our primitive count.
Why not just keep all points, as UniSplat's dual branch does? Keeping every point reintroduces the redundancy that voxel alignment removed. The dense-query baseline measures this trade-off. Selection is justified only if, at equal primitive count, it beats dense querying on thin structures without losing PSNR.
Proposed Experiments
The first experiment is a fixed-budget sweep on ScanNet++. We vary the query budget $K$ from 0% to 20% of occupied voxels and plot thin-structure recall, Chamfer distance and HRRC PSNR against primitive count and memory. The finer-grid, SPLATT3RFUSION-sweep and dense-query curves go on the same axes. The decisive check is that our curve lies above all three in the region of fewer primitives than VolSplat at native resolution.
The second experiment is the source-of-information check. Image-feature queries should beat interpolated-voxel-feature queries by a wide margin on structures thinner than one voxel, and by little on thicker ones. The gap should grow as voxels get coarser.
The third is the masking check. The improvement over VolSplat should be small at 256×256 PSNR on RealEstate10K and large at HRRC resolutions and in geometric metrics. Confirming this would support SurfSplat's observation that low-resolution scores hide missing geometry.
The fourth tests the allocation itself. We correlate predicted scores $s_v$ with measured coarse-model error on held-out scenes, and compare the random-allocation ablation against the error-driven one at equal $K$.
The fifth tests cross-dataset generalization. We train on ScanNet++ and RealEstate10K and evaluate zero-shot on ACID, where no reported scalar was found on Lacuna for the competing models, so all numbers there will be produced by our own runs.
The sixth stresses robustness. We perturb input camera poses with small rotations and compare visibility-gated attention against hard projection, expecting the attention reader to degrade more gracefully, as SGFormer's stress tests suggest.
References
- VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction
- SurfSplat: Conquering Feedforward 2D Gaussian Splatting with Surface Continuity Priors
- FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstruction
- SGFormer: Semantic-Geometry Fusion Transformer for Multi-modal 3D Panoptic Segmentation
- PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
- MVX-Net: Multimodal VoxelNet for 3D Object Detection
- UniSplat: Unified Spatio-Temporal Fusion via 3D Latent Scaffolds for Dynamic Driving Scene Reconstruction
- Learning Compact 3D Gaussians via Feed-Forward Point Fusion
- Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding
- ARKitSceneRefer: Text-based Localization of Small Objects in Diverse Real-World 3D Indoor Scenes
- ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes