A survey on surface reconstruction based on 3D Gaussian splatting

Zheng Xu, Gang Chen, Feng Li, Lingyu Chen, Yuanhang ChengPeerJ 2025 · DOI

TL;DR

This survey provides a comprehensive overview of the rapidly evolving field of 3D surface reconstruction using 3D Gaussian Splatting (3D GS). It offers a structured roadmap of methodologies, datasets, and applications, highlighting the advancements and challenges in this area, and serves as a valuable resource for researchers.

What Problem Does This Paper Address?

3D surface reconstruction, a cornerstone of computer graphics and vision, has seen a surge in innovation with the advent of techniques like Neural Radiance Fields (NeRF) and, more recently, 3D Gaussian Splatting (3D GS). While these methods enable impressive 3D model generation and rendering, the rapid pace of development makes it challenging to keep track of new algorithms and their underlying principles. This paper addresses this challenge by providing a structured survey that consolidates the core methodologies, categorizes reconstruction approaches, and outlines the current landscape of 3D GS-based surface reconstruction, including its applications and future directions.

Methods

The survey focuses on 3D Gaussian Splatting (3D GS) as the central framework for surface reconstruction. It breaks down the process into three key phases:

  • Scene Representation: This involves how 3D scenes are represented using 3D Gaussians. The paper discusses both static and dynamic scene representations, exploring techniques that use implicit neural surfaces, explicit primitives, and various ways to adapt 3D Gaussians (e.g., as 2D planes, quadric surfaces, or within tetrahedral grids) to better approximate scene geometry.
  • Gaussian Optimization: This phase details how the parameters of the 3D Gaussians are optimized during training to achieve accurate surface fitting. It covers various loss functions and regularization constraints that incorporate geometric attributes like depth, normals, and opacity, as well as semantic information.
  • Surface Structure Extraction: This involves extracting the final 3D surface mesh from the optimized Gaussian representation. The survey discusses methods like Poisson reconstruction and TSDF (Truncated Signed Distance Function) based approaches, as well as techniques for handling dynamic scenes and extracting deformable surfaces.

Datasets

The paper categorizes datasets into two main types:

  • Real Scene Datasets: These capture complex real-world environments and include:
  • Tanks and Temples
    • DTU
    • Replica
    • RGBD-SLAM Benchmark
    • ScanNet
    • ScanNet++
    • TUM-RGBD
  • Synthetic Scene Datasets: These are generated through simulations and offer advantages in cost, scalability, and quality. Examples include:
  • Mip-NeRF360
    • Blended-MVS
    • NeRF Synthetic
    • Matrix City

The paper also includes a table summarizing typical datasets for 3D reconstruction and another table comparing the performance metrics of various surface reconstruction methods on benchmark datasets.

Key Insights and Ideas

  • 3D GS as a Paradigm Shift: The emergence of 3D GS has significantly improved rendering performance and redefined scene representations, offering a powerful alternative to traditional methods and even NeRFs in certain aspects.
  • Bridging Implicit and Explicit Representations: Many recent methods aim to combine the strengths of implicit representations (like SDFs) with the efficiency of explicit Gaussian representations to achieve high-fidelity reconstructions.
  • The Importance of Constraints: Effective surface reconstruction heavily relies on carefully designed optimization constraints and regularization terms that leverage geometric priors, semantic information, and multi-view consistency.
  • Dynamic Scenes Remain a Challenge: While progress has been made, accurately representing and reconstructing deformable objects and dynamic scenes using 3D GS still presents significant challenges, particularly in terms of computational cost and temporal consistency.
  • Dataset Quality is Crucial: The survey highlights the limitations of current datasets, emphasizing the need for larger, more diverse, and fine-grained datasets that capture complex environmental conditions and object variations to improve the generalization ability of reconstruction algorithms.

Key Results

The paper is a survey, so it doesn't present new experimental results but rather synthesizes findings from numerous existing studies. It highlights that 3D GS methods, when integrated with rasterization, achieve faster rendering speeds than NeRFs. The survey categorizes various approaches and discusses their effectiveness in achieving high-fidelity geometric reconstruction and rendering quality across different applications like autonomous driving and digital human modeling. Performance metrics from benchmark datasets (DTU, Tanks and Temples) are presented, showing the comparative strengths of different methods in terms of reconstruction speed, accuracy (F1 score), and quality (PSNR).

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Dataset Limitations: Existing datasets are often limited in quantity, quality, and diversity. Future work should focus on developing large-scale, fine-grained 3D datasets that capture complex scene variations under different lighting and material conditions.
  • Scalability Challenges: Techniques designed for small-scale scenes often perform poorly when scaled to large environments. There's a need for restructured Gaussian organization and intelligent batch modeling strategies for efficient and coherent reconstruction of urban environments.
  • Geometric Detail Quality: Current methods can struggle to accurately represent fine geometric features, especially with fixed-resolution training data. Future research should explore variable-resolution training pipelines and integrate image upscaling for improved fidelity, even in suboptimal imaging conditions.
  • Balancing Fidelity and Efficiency: While numerous constraints improve reconstruction quality, they can increase computational load. Future work should focus on integrated optimization frameworks that balance reconstruction fidelity with computational cost.
  • Dynamic Scene Complexity: Representing and reconstructing deformable objects in dynamic scenes remains a significant challenge, requiring further research into efficient and robust methods.

Summary updated

Abstract

Surface reconstruction is a foundational topic in computer graphics and has gained substantial research interest in recent years. With the emergence of advanced neural radiance fields (NeRFs) and 3D Gaussian splatting (3D GS), numerous innovative many novel algorithms for 3D model surface reconstruction have been developed. The rapid expansion of this field presents challenges in tracking ongoing advancements. This survey aims to present core methodologies for the surface reconstruction of 3D models and establish a structured roadmap that encompasses 3D representations, reconstruction methods, datasets, and related applications. Specifically, we introduce 3D representations using 3D Gaussians as the central framework. Additionally, we provide a comprehensive overview of the rapidly evolving surface reconstruction methods based on 3D Gaussian splatting. We categorize the primary phases of surface reconstruction algorithms for 3D models into scene representation, Gaussian optimization, and surface structure extraction. Finally, we review the available datasets, applications, and challenges and suggest potential future research directions in this domain. Through this survey, we aim to provide valuable resources that support and inspire researchers in the field, fostering advancements in 3D reconstruction technologies.
BibTeX
@inproceedings{xu2025survey,
  author = {Zheng Xu and Gang Chen and Feng Li and Lingyu Chen and Yuanhang Cheng},
  title = {{A survey on surface reconstruction based on 3D Gaussian splatting}},
  booktitle = {PeerJ},
  year = {2025},
  doi = {10.7717/peerj-cs.3034},
}
Figure