PhGS: Post-Hoc Pruning and Refinement of Single-View Feed-Forward 3D Gaussian Reconstructions

Rinto Yagawa, Han Cheng, Dieter Schmalstieg, Hideo Saito, Shohei MoriarXiv 2026 · arXiv

TL;DR

Single-view feed-forward 3D Gaussian Splatting (3DGS) models suffer from severe spatial redundancy because they predict a fixed number of Gaussians for every camera ray. PhGS introduces a plug-and-play, post-hoc framework that prunes redundant Gaussians using a novel importance score and refines the remaining ones with a lightweight recurrent module. It cuts memory usage and Gaussian counts by 50% to 75% with negligible impact on rendering quality, while boosting novel-view rendering speeds by over 56%.

What Problem Does This Paper Address?

Feed-forward 3D Gaussian Splatting has revolutionized 3D reconstruction by generating explicit 3D scenes from images in a single forward pass. However, when dealing with a single input image, networks cannot rely on multi-view consistency to resolve occlusions. To compensate, state-of-the-art single-view models predict a fixed number of Gaussians ($K > 1$) along every single camera ray.

This "K-Gaussians-per-ray" strategy creates massive spatial redundancy. Flat walls or uniform backgrounds end up packed with unnecessary geometric primitives, inflating memory consumption and slowing down rendering. While compaction methods exist for multi-view setups, they are tightly coupled with multi-view architectures and cannot be easily adapted to single-view models. PhGS solves this by introducing a backbone-agnostic approach that operates entirely on the output space of any pre-trained single-view model without requiring retraining.

Methods

The PhGS pipeline consists of two main phases: importance-score-based pruning and recurrent refinement.

Figure 2. Overview of our cross-model post-hoc compaction pipeline. The pipeline operates in two phases. (a) We compute a spatial importance map and probabilistically sample a mask to prune redundant primitives (Sec. 3.1). (b) The surviving Gaussians undergo iterative updates to recover rendering quality (Sec. 3.2). At each step, the recurrent network predicts parameter updates (∆Gt, ∆Zt) from the current Gaussian parameters Gt, the hidden state Zt, and the rendering error feature Et.
  1. Importance-Score-Based Pruning: Instead of randomly dropping primitives, PhGS computes an importance score for each Gaussian by combining an edge prior from the input image (using a Laplacian operator) with the predicted opacity of the Gaussian. Primitives with high opacity or those aligned with structural boundaries are prioritized. A binary mask is then probabilistically sampled to drop a user-specified fraction of the Gaussians.
  2. Recurrent Refinement Module: Pruning naturally leaves gaps in the 3D scene. To restore visual fidelity, the remaining Gaussians are passed through a lightweight, two-layer MLP. This module iteratively predicts additive updates to the Gaussians' parameters (opacity, scale, rotation, and position). To stay scalable and avoid expensive spatial operations like kNN or global attention, the MLP processes each Gaussian independently while incorporating a global-context feature via average pooling.
  3. Keep Ratio Mix (krMix): To prevent having to train a separate refinement network for every desired compression level, the authors introduce a training strategy called krMix. By randomly sampling the target keep ratio during training, a single model learns to dynamically support arbitrary keep ratios (e.g., 25%, 50%, 75%) at inference time.

Datasets

The authors trained and evaluated their method using the following benchmarks:

  • RealEstate10K: Used for primary in-domain training and quantitative evaluation.
  • KITTI & DL3DV: Used to evaluate the zero-shot cross-domain generalization capabilities of the refinement module on outdoor driving scenes and diverse real-world environments, respectively.

Key Insights and Ideas

  • Decoupling is Feature-Rich: By treating the underlying feed-forward network as a frozen black box, PhGS avoids the massive computational burden of retraining heavy base models. It can be applied retroactively to any architecture (e.g., Flash3D, Niagara, SHARP).
  • "Inflating" Surviving Primitives: Statistical analysis reveals that the recurrent refinement module compensates for missing geometry primarily by shifting the opacity and scaling distributions of the surviving Gaussians to the right. Essentially, the remaining Gaussians become slightly larger and more opaque to fill the gaps left by pruned neighbors.
  • Asymmetric Generalization: The authors discovered that a refinement module trained at a fixed 50% keep ratio generalizes well when asked to prune more aggressively (25%), but fails dramatically when pruning less aggressively (75%). The krMix strategy successfully mitigates this issue, ensuring stable performance across all ratios.

Key Results

The authors evaluated PhGS across three distinct backbones on the RealEstate10K dataset at a 50% keep ratio ($r=0.5$):

  • Fidelity Preservation: For Flash3D, the PSNR dropped by a mere 0.06 dB while cutting the Gaussian count in half. For SHARP, which produces an exceptionally dense native point cloud, PhGS actually improved the PSNR by 0.30 dB while reducing storage from 66.06 MB to 33.03 MB.
  • Throughput Boost: On an NVIDIA RTX 4070 Ti, compressing Flash3D reconstructions by 50% increased the novel-view rendering throughput from 1,443 views per second to 2,257 views per second—a 56.4% speedup.
  • Baselines Outperformed: PhGS consistently outperformed existing multi-view post-hoc pruning methods like LightGaussian and PUP 3D-GS adapted to the single-view setting, demonstrating the value of its edge-opacity scoring function.
Figure 1. Our post-hoc compaction method seamlessly integrates with any single-view feed-forward 3D Gaussian Splatting model. Pre- trained baselines such as Flash3D [28], Niagara [32] and SHARP [20] predict dense Gaussians; our method prunes redundant primitives and refines the remaining ones, achieving arbitrary memory reduction while largely preserving or even improving the novel-view rendering fidelity. Left: performance across backbones, with LightGaussian [7] and PUP 3D-GS [10] applied to Flash3D [28] as post-hoc pruning methods; right: rendering quality with Flash3D [28]. The insets show the Gaussian center points.

Limitations and Future Work

While PhGS significantly accelerates downstream novel-view rendering, the initial full-inference pipeline experiences a 35% runtime overhead due to the iterative unrolling of the recurrent refinement module (adding roughly 76 ms per scene).

Additionally, because the recurrent module only receives error feedback from the single source view during inference, its ability to correct geometric artifacts that only become visible from extreme, highly oblique novel viewpoints remains limited. Future work could explore incorporating self-supervised geometric priors to better guide the refinement process from unseen angles.

Summary updated

Abstract

Recent single-view feed-forward 3D Gaussian Splatting (3DGS) generation predicts a fixed number of Gaussians per camera ray, introducing severe spatial redundancy. Most existing compaction strategies target multi-view setups to exploit cross-view consistency and are incompatible with single-image models. Instead of retraining the base feed-forward network to directly output compact representations, our insight is to keep the base models frozen and apply post-hoc pruning and recurrent refinement to the generated Gaussians. Consequently, we propose a backbone-agnostic compaction pipeline for single-view feed-forward 3DGS that couples an importance-score-based pruning mechanism with a trainable, lightweight recurrent refinement module, which iteratively updates the surviving primitives to restore image quality. Our results demonstrate seamless integration with existing baselines while preserving novel-view rendering fidelity and achieving high memory reduction. Furthermore, our method supports flexible inference-time keep ratios for application needs.
BibTeX
@misc{yagawa2026phgs,
  author = {Rinto Yagawa and Han Cheng and Dieter Schmalstieg and Hideo Saito and Shohei Mori},
  title = {{PhGS: Post-Hoc Pruning and Refinement of Single-View Feed-Forward 3D Gaussian Reconstructions}},
  year = {2026},
  eprint = {2609.20623},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.20623},
}
Figure