TL;DR
CrossPressor is a training-free "plug-and-play" module that enables feed-forward 3D Gaussian Splatting (3D-GS) to scale to hundreds of input views. By introducing "coverage-cycled evidence routing," it achieves up to a 26x speedup in cross-view fusion and a 73% reduction in peak fusion memory while maintaining near-perfect reconstruction quality.
What Problem Does This Paper Address?
Feed-forward 3D Gaussian Splatting (3D-GS) is a powerful paradigm for generating 3D scenes from a few images in a single forward pass. However, these models struggle to scale. As you increase the number of input views to improve reconstruction quality, the computational cost and memory usage explode.
Recent "anchor-based" methods like ZPressor attempted to fix this by decoding Gaussians from a fixed, compact set of "anchor" feature maps. While this keeps the final number of 3D Gaussians constant, the authors of this paper identified a hidden bottleneck: the representation–fusion mismatch. Even if the output representation is fixed, the process of fusing information from hundreds of "support" views into those anchors still scales linearly with the number of views. In long sequences, the fusion step becomes the dominant cost, making real-time processing impossible.
Methods
CrossPressor introduces a training-free method to bound the fusion cost without losing the "evidence" provided by additional views. Its core innovation is coverage-cycled evidence routing:
- Spatial Striding: Instead of using every pixel from every support view, CrossPressor selects a sparse grid of features. As more views are added, the grid becomes coarser to keep the total number of processed tokens constant.
- Cycled Offsets: To ensure no information is permanently lost, the spatial sampling offset is varied across different support views and across different fusion layers. For example, Layer 1 might look at the top-left pixels of View A, while Layer 2 looks at the bottom-right. Over the course of the network, the anchors "see" the entire scene.
- Pre-projection Routing: Unlike standard token reduction methods that prune tokens after expensive linear projections, CrossPressor routes raw features before the Key/Value (K/V) projections in the Transformer blocks. This directly slashes the most expensive parts of the attention mechanism.
- Joint Context Packing: Routed features are packed into a single compact sequence, allowing the model to use standard, optimized attention kernels without modification.
Datasets
The authors evaluated CrossPressor on two primary benchmarks for novel view synthesis:
- RealEstate10K (RE10K): A large-scale dataset of indoor and outdoor walk-throughs, used here to test long-context scaling with up to 256 input views.
- ACID: Used to test the model's ability to generalize to new types of scenes (cross-dataset generalization) without any retraining.
Key Insights and Ideas
- The "Aha" Moment on Mismatch: The realization that bounding the output size of a model is insufficient for scalability if the input processing remains unbounded.
- Spatial vs. View-wise Reduction: Most pruning methods drop entire images or random tokens. CrossPressor keeps every camera represented but reduces the spatial density of each. This "view-normalized" approach is much more robust for 3D reconstruction, where every viewpoint might contain unique geometric clues.
- Temporal Integration via Layers: By cycling the sampling pattern across layers, the network acts like a scanner, integrating different spatial details as the signal propagates deeper. This allows a sparse model to achieve "dense-like" quality.
Key Results
The quantitative gains are substantial, especially as the number of input views increases:
- Massive Fusion Speedup: At 256 input views, CrossPressor is 26.12× faster in cross-view fusion compared to the "Exact" dense baseline.
- End-to-End Efficiency: This translates to a 1.43× speedup for the entire reconstruction pipeline.
- Memory Savings: It reduces the incremental peak memory required for fusion by 73.8% (from ~653 MiB down to ~171 MiB on MVSplat).
- Quality Preservation: Despite processing only ~1.85% of the total possible query-key interactions, CrossPressor maintains a PSNR within 0.006 dB of the dense model. It significantly outperforms other training-free baselines like ToMe and Zero-TPrune in maintaining quality on "difficult" scenes (measured by the p5 $\Delta$PSNR metric).
Limitations and Future Work
- Per-View Overhead: While fusion is bounded, initial per-view operations (like the image encoder/feature extraction) still scale with the number of views. Future work could explore more scalable encoders.
- Metadata Scaling: The metadata for routing (indices and offsets) still grows with the view count, though its computational footprint is currently negligible compared to the neural network layers.
- Fixed Strides: The current method uses power-of-two strides. A more flexible or learned routing policy might offer even better quality-efficiency trade-offs, though it would likely require retraining.
Summary updated