Sparse2DGS: Sparse-View Surface Reconstruction Using 2D Gaussian Splatting with Dense Point Cloud

Nobuyuki Takama, Shintaro Ito, Koichi Ito, Hwann-Tzong Chen, Takafumi Aoki2025 · DOI

Abstract

Sparse2DGS is a novel 3D surface reconstruction method that generates highly accurate 3D meshes from just three input images. By combining a deep learning stereo model (DUSt3R) with traditional Multi-View Stereo (COLMAP) to create a dense initial point cloud, it overcomes the poor initialization problem that usually plagues Gaussian Splatting in sparse-view scenarios. Sparse2DGS achieves state-of-the-art accuracy in under two minutes, significantly outperforming existing NeRF-based methods in efficiency while maintaining comparable geometric precision.

Introduction

Reconstructing a 3D object from 2D images—known as Multi-View Stereo (MVS)—traditionally requires dozens or hundreds of overlapping photos. While newer techniques like Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) have revolutionized novel view synthesis, they still struggle when given only a handful of images (e.g., three).

For Gaussian Splatting specifically, the optimization process relies heavily on an initial sparse point cloud generated by Structure from Motion (SfM). When only three images are available, SfM produces very few 3D points. Without enough starting points, the Gaussians cannot properly cover the object, resulting in failed optimizations and poor 3D geometry. While some NeRF-based methods can handle sparse views, they often require massive datasets and days of pre-training. Sparse2DGS addresses this by achieving fast, accurate 3D reconstruction from just three images without extensive pre-training.

Methods

The core novelty of Sparse2DGS lies in its initialization pipeline, which builds upon 2D Gaussian Splatting (2DGS). Unlike standard 3DGS, 2DGS uses flat 2D ellipses, making it better suited for extracting solid surfaces.

The Sparse2DGS pipeline integrates DUSt3R and COLMAP for dense initialization followed by 2D Gaussian optimization.

The pipeline consists of several key steps:

  1. Dense Point Cloud Generation: The system feeds three input images into DUSt3R (a foundation model for stereo vision) and COLMAP MVS.
  2. Point Cloud Integration: DUSt3R provides dense geometry but may have coordinate errors, while COLMAP offers high metric accuracy but sparse coverage. The authors align the DUSt3R cloud with the COLMAP cloud using the Iterative Closest Point (ICP) algorithm.
  3. Gaussian Optimization: This fused, dense point cloud serves as the starting position for 2D Gaussians. These are optimized using color loss, depth distortion loss, and normal consistency loss.
  4. No Densification: Because the initial point cloud is already dense, the standard Gaussian densification step (cloning/splitting) is skipped, significantly speeding up optimization.
  5. Mesh Extraction: Depth maps rendered from the optimized Gaussians are fused into a 3D mesh using Truncated Signed Distance Function (TSDF) integration.

Experimental Setup

The method was evaluated on the DTU dataset, using 15 scenes resized to 800x600. To simulate a challenging sparse-view scenario, the authors selected 3-image subsets with minimal overlap. Scans containing highly reflective objects (scans 69, 97, and 110) were excluded, as specular highlights break the multi-view consistency assumptions required for accurate reconstruction.

Key Results

Sparse2DGS demonstrates a significant leap in both accuracy and efficiency for sparse-view reconstruction.

Qualitative comparison of Sparse2DGS against state-of-the-art methods on DTU scans.
  • Accuracy: Sparse2DGS achieved a mean Chamfer Distance of 1.51 on the DTU dataset, tying for first place with UFORecon, a state-of-the-art NeRF-based method.
  • Speed: On a single RTX 4090 GPU, optimization takes approximately 100 seconds, and mesh reconstruction takes 10 seconds. This is a massive improvement over UFORecon, which requires days of pre-training to achieve similar results.
  • Ablation Study: The custom initialization is critical. Using standard SfM initialization resulted in a poor Chamfer Distance of 2.54, whereas the fused DUSt3R + COLMAP initialization reduced the error to 1.51.

Discussion and Conclusion

The primary insight of this work is that the failure of Gaussian Splatting in sparse views is primarily an initialization bottleneck, not a limitation of the renderer. By combining the dense estimation capabilities of DUSt3R with the metric precision of COLMAP, Sparse2DGS provides the necessary geometric anchors for stable optimization. Furthermore, skipping the densification phase makes the process incredibly fast.

Despite its success, the method struggles with highly reflective surfaces. Additionally, the current pipeline relies on ground-truth camera parameters. Future work aims to jointly optimize camera poses and incorporate view-dependent effects to handle complex materials and uncalibrated image sets.

Summary updated

Abstract

Gaussian Splatting (GS) has gained attention as a fast and effective method for novel view synthesis. It has also been applied to 3D reconstruction using multi-view images and can achieve fast and accurate 3D reconstruction. However, GS assumes that the input contains a large number of multi-view images, and therefore, the reconstruction accuracy significantly decreases when only a limited number of input images are available. One of the main reasons is the insufficient number of 3D points in the sparse point cloud obtained through Structure from Motion (SfM), which results in a poor initialization for optimizing the Gaussian primitives. We propose a new 3D reconstruction method, called Sparse2DGS, to enhance 2DGS in reconstructing objects using only three images. Sparse2DGS employs DUSt3R, a fundamental model for stereo images, along with COLMAP MVS to generate highly accurate and dense 3D point clouds, which are then used to initialize 2D Gaussians. Through experiments on the DTU dataset, we show that Sparse2DGS can accurately reconstruct the 3D shapes of objects using just three images.
BibTeX
@misc{takama2025sparse,
  author = {Nobuyuki Takama and Shintaro Ito and Koichi Ito and Hwann-Tzong Chen and Takafumi Aoki},
  title = {{Sparse2DGS: Sparse-View Surface Reconstruction Using 2D Gaussian Splatting with Dense Point Cloud}},
  year = {2025},
  doi = {10.1109/icip55913.2025.11084702},
}
Figure