Efficient Construction of Implicit Surface Models From a Single Image for Motion Generation

Wei-Teng Chu, Tianyi Zhang, Matthew Johnson-Roberson, Weiming Zhi

Abstract

FINS (Fast Image-to-Neural Surface) is a lightweight framework designed to reconstruct high-fidelity 3D Signed Distance Fields (SDFs) from a single RGB image in approximately 10 seconds. By leveraging 3D foundation models and a hybrid second-order optimization strategy using K-FAC, FINS enables robots to generate smooth, surface-following motions in real-time. This approach overcomes the data-intensive and time-consuming nature of traditional neural surface reconstruction methods, making it suitable for reactive robotic control.

Introduction

For autonomous robots to navigate or interact with objects—such as painting a car or inspecting a turbine—they require a precise mathematical map of their surroundings. Signed Distance Fields (SDFs) are the gold standard for this, as they provide the exact distance from any point in space to the nearest surface. However, traditional methods for building these neural maps, such as NeuS, suffer from high data requirements (dozens of images) and long training times (minutes to hours). FINS addresses these challenges by "lifting" a single 2D image into a complete, mathematically consistent 3D representation fast enough for real-time deployment.

Methodology

The FINS framework achieves its speed and accuracy through three primary components:

  1. Foundation Model Lifting: Instead of learning geometry from scratch, FINS utilizes pre-trained 3D foundation models (e.g., DUSt3R or VGGT) to estimate an initial point cloud and confidence map from a single RGB image.
  2. Efficient Architecture: The model employs a multi-resolution hash grid encoder, allowing for compact storage and rapid access to geometric details during training.
  3. Hybrid Optimization: FINS introduces a staged optimization approach. It begins with the Lion optimizer for initial shape formation (Stage 1) and then switches to K-FAC (Kronecker-Factored Approximate Curvature), an approximate second-order optimizer, for the final 40% of training (Stage 2). K-FAC accounts for the curvature of the optimization landscape, enabling the model to achieve high-precision detail in seconds.

Results and Evaluation

The authors evaluated FINS using the DTU and BlendedMVS datasets. FINS demonstrates significant improvements in efficiency, converging in approximately 10 seconds on a consumer-grade laptop GPU (RTX 4060). In contrast, multi-view methods like NeuS require over 4 minutes and dozens of images to achieve comparable results.

Reconstruction results across diverse object categories from the DTU and BlendedMVS datasets, showing the fidelity achieved from a single view.

Robotic Utility and SDF Quality

A critical insight of this work is that robotics applications require more than just a visually appealing mesh; they require a valid SDF gradient (the Eikonal property). If the distance field is noisy away from the surface, motion planners will fail. FINS ensures that the reconstructed SDFs are smooth and accurate enough for reactive control. This was demonstrated using a Franka Emika Panda robot arm successfully tracing the surface of a reconstructed statue in simulation.

Visualization of reconstructed surfaces, motion paths (red lines), and SDF heatmaps demonstrating the smoothness and gradient consistency required for robotic control.

Conclusion and Future Work

While FINS provides a rapid path to 3D reconstruction, it remains dependent on the quality of the initial foundation model priors and faces inherent ambiguities in single-view reconstruction. Future research will focus on "evolving" representations, allowing robots to continuously update their SDFs as they gather more visual data, effectively performing real-time SLAM with high-fidelity surfaces.

Summary updated

Abstract

Implicit representations have been widely applied in robotics for obstacle avoidance and path planning. In this paper, we explore the problem of constructing an implicit distance representation from a single image. Past methods for implicit surface reconstruction, such as \emph{NeuS} and its variants generally require a large set of multi-view images as input, and require long training times. In this work, we propose Fast Image-to-Neural Surface (FINS), a lightweight framework that can reconstruct high-fidelity surfaces and SDF fields based on a single or a small set of images. FINS integrates a multi-resolution hash grid encoder with lightweight geometry and color heads, making the training via an approximate second-order optimizer highly efficient and capable of converging within a few seconds. Additionally, we achieve the construction of a neural surface requiring only a single RGB image, by leveraging pre-trained foundation models to estimate the geometry inherent in the image. Our experiments demonstrate that under the same conditions, our method outperforms state-of-the-art baselines in both convergence speed and accuracy on surface reconstruction and SDF field estimation. Moreover, we demonstrate the applicability of FINS for robot surface following tasks and show its scalability to a variety of benchmark datasets.
BibTeX
@misc{chu2025efficient,
  author = {Wei-Teng Chu and Tianyi Zhang and Matthew Johnson-Roberson and Weiming Zhi},
  title = {{Efficient Construction of Implicit Surface Models From a Single Image for Motion Generation}},
  year = {2025},
  eprint = {2509.20681},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.20681},
}
Figure