Abstract
FINS (Fast Image-to-Neural Surface) is a lightweight framework designed to reconstruct high-fidelity 3D Signed Distance Fields (SDFs) from a single RGB image in approximately 10 seconds. By leveraging 3D foundation models and a hybrid second-order optimization strategy using K-FAC, FINS enables robots to generate smooth, surface-following motions in real-time. This approach overcomes the data-intensive and time-consuming nature of traditional neural surface reconstruction methods, making it suitable for reactive robotic control.
Introduction
For autonomous robots to navigate or interact with objects—such as painting a car or inspecting a turbine—they require a precise mathematical map of their surroundings. Signed Distance Fields (SDFs) are the gold standard for this, as they provide the exact distance from any point in space to the nearest surface. However, traditional methods for building these neural maps, such as NeuS, suffer from high data requirements (dozens of images) and long training times (minutes to hours). FINS addresses these challenges by "lifting" a single 2D image into a complete, mathematically consistent 3D representation fast enough for real-time deployment.
Methodology
The FINS framework achieves its speed and accuracy through three primary components:
- Foundation Model Lifting: Instead of learning geometry from scratch, FINS utilizes pre-trained 3D foundation models (e.g., DUSt3R or VGGT) to estimate an initial point cloud and confidence map from a single RGB image.
- Efficient Architecture: The model employs a multi-resolution hash grid encoder, allowing for compact storage and rapid access to geometric details during training.
- Hybrid Optimization: FINS introduces a staged optimization approach. It begins with the Lion optimizer for initial shape formation (Stage 1) and then switches to K-FAC (Kronecker-Factored Approximate Curvature), an approximate second-order optimizer, for the final 40% of training (Stage 2). K-FAC accounts for the curvature of the optimization landscape, enabling the model to achieve high-precision detail in seconds.
Results and Evaluation
The authors evaluated FINS using the DTU and BlendedMVS datasets. FINS demonstrates significant improvements in efficiency, converging in approximately 10 seconds on a consumer-grade laptop GPU (RTX 4060). In contrast, multi-view methods like NeuS require over 4 minutes and dozens of images to achieve comparable results.
Robotic Utility and SDF Quality
A critical insight of this work is that robotics applications require more than just a visually appealing mesh; they require a valid SDF gradient (the Eikonal property). If the distance field is noisy away from the surface, motion planners will fail. FINS ensures that the reconstructed SDFs are smooth and accurate enough for reactive control. This was demonstrated using a Franka Emika Panda robot arm successfully tracing the surface of a reconstructed statue in simulation.
Conclusion and Future Work
While FINS provides a rapid path to 3D reconstruction, it remains dependent on the quality of the initial foundation model priors and faces inherent ambiguities in single-view reconstruction. Future research will focus on "evolving" representations, allowing robots to continuously update their SDFs as they gather more visual data, effectively performing real-time SLAM with high-fidelity surfaces.
Summary updated