SPC-GS: Gaussian Splatting with Semantic-Prompt Consistency for Indoor Open-World Free-view Synthesis from Sparse Inputs

Guibiao Liao, Qing Li, Zhenyu Bao, Guoping Qiu, Kanglin LiuCVPR 2025 · DOI

Introduction

3D Gaussian Splatting (3DGS) has significantly advanced real-time novel view synthesis; however, it remains highly dependent on dense input data. In indoor "inside-out" scenarios where camera overlap is naturally low, 3DGS often fails when provided with sparse inputs (e.g., as few as 12 images). These failures typically manifest as sparse point cloud initialization and insufficient supervision, leading to "floaters," blurry artifacts, and semantic ambiguity in unobserved regions.

SPC-GS addresses these challenges by integrating generative video models for robust initialization and leveraging Segment Anything Model 2 (SAM2) to enforce semantic consistency across views.

Methodology

Scene-layout-based Gaussian Initialization (SGI)

Standard Structure-from-Motion (SfM) often fails to produce a dense point cloud from sparse indoor snapshots. SPC-GS introduces SGI, which utilizes a Video Generation Model (VGM), specifically MotionCtrl, to "hallucinate" additional perspectives. By generating short video clips (pan, tilt, zoom) for each training image, the system creates a much denser initial 3D point cloud. This process is further refined using an Outlier Gaussian Primitive Removal (OGR) strategy to eliminate geometric noise.

Semantic-Prompt Consistency (SPC) Regularization

To maintain semantic coherence in "open-world" scenarios, SPC-GS employs SAM2 as a consistency teacher. The method uses Iterative Stochastic Prompting (ISP) to track object masks between known views and rendered pseudo-views from unobserved angles. By applying 2D and 3D semantic constraints—including region boundary erosion to prevent feature "bleeding"—the model ensures that objects like furniture and architectural elements are correctly identified from any viewpoint.

Experimental Results

The researchers evaluated SPC-GS on the Replica and ScanNet datasets using a strict 12-view protocol.

Comprehensive comparison of SPC-GS against the Gau-Grouping baseline. (a1, b1) show the dramatic increase in initial point density using SGI. (a2, b2) compare rendered RGB and semantic segmentation results. (c) provides quantitative metrics showing superior performance in PSNR, SSIM, and mIoU.

Quantitative Performance

SPC-GS sets a new state-of-the-art for sparse-input indoor scenes. On the Replica dataset, it achieved a PSNR of 22.011 dB, a significant improvement over previous methods. Furthermore, the model demonstrated a 7.3% increase in mean Intersection over Union (mIoU) for open-world segmentation compared to Gau-Grouping, highlighting its superior semantic understanding.

Quantitative comparison of PSNR, SSIM, and mIoU metrics between Gau-Grouping and the proposed SPC-GS method.

Conclusion

SPC-GS effectively bridges the gap between sparse data and high-quality 3D reconstruction. By treating generative AI as a geometric prior and SAM2 as a semantic bridge, the framework achieves superior visual fidelity and robust object recognition. While currently limited to static scenes and dependent on the physical plausibility of video generation models, SPC-GS represents a major step toward practical, sparse-view indoor scene understanding.

Summary updated

BibTeX
@inproceedings{liao2025spc,
  author = {Guibiao Liao and Qing Li and Zhenyu Bao and Guoping Qiu and Kanglin Liu},
  title = {{SPC-GS: Gaussian Splatting with Semantic-Prompt Consistency for Indoor Open-World Free-view Synthesis from Sparse Inputs}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.01052},
}
Figure