Abstract
Bayesian EKF (Extended Kalman Filter) is cross-fertilized with Navier's equations solid deformation modeling to compute 3D non-rigid structure from monocular camera motion. The method operates with a projective camera and autonomously computes -for every sequence frame- both the geometry and the matches. The combination results in an sequential efficient method to code the rich available physical priors, particularly relevant for medical monocular endoscope sequences such as those observing the abdominal cavity. The scene is modeled as a planar triangular mesh where each triangular element is modeled as a thin-plate. Navier's equations are solved numerically by means of FEM (Finite Element Method), being the FEM nodes the 3D points of the estimated sparse structure. Despite the assumed elastic model is only valid for small deformations, the eventually large scene deformation is accurately computed, because the EKF extracts partial measurements of the actual scene deformation at frame rate. Ground truth is computed from a real sequence gathered with hand-held stereo camera. The observed non-rigid scene is a silicone cloth fixed on a stretcher. It is deformed under the action of an unknown force applied on the silicone surface. It is shown how the estimation resulting from applying the proposed algorithm a monocular sequence is statistically consistent with the ground truth.
Introduction
Standard Simultaneous Localization and Mapping (SLAM) algorithms are typically built on a "rigid world" assumption, presuming that the environment remains static. While effective for applications like indoor robotics, this assumption fails in dynamic environments such as the human body, where tissues shift and deform during medical procedures. Existing techniques for Non-Rigid Structure from Motion (NRSfM) often rely on batch processing or simplified linear models that lack physical grounding. This research addresses the challenge of creating a sequential, real-time system that utilizes physical laws to constrain realistic deformations, enhancing the robustness of 3D reconstruction from a single moving camera.
Methodology
The proposed approach integrates two mathematical frameworks: the Extended Kalman Filter (EKF) and the Finite Element Method (FEM). The EKF serves as the central estimator, maintaining a state vector of camera motion and 3D node coordinates. The FEM provides physical constraints by modeling the scene as a planar triangular mesh of "thin-plate" elements governed by Navier’s equations for linear elasticity.
A key innovation is modeling unknown external forces as stochastic process noise within the EKF. Although linear elastic models are strictly valid only for small deformations, the high frame rate (30Hz) of the EKF allows the system to accurately track large total deformations by accumulating small, physically valid incremental steps. Furthermore, material properties like Young’s modulus are factored out through parameter normalization, simplifying filter tuning.
Experimental Setup
Validation was conducted using a custom experimental setup featuring a textureless silicone cloth marked with artificial points and secured in a circular stretcher. Significant non-rigid motion was induced by applying unknown forces with a stick. While the algorithm processed monocular data, a hand-held mobile stereo rig provided high-accuracy 3D ground truth for comparison, processed via commercial photogrammetric software.
Results and Discussion
The system demonstrated statistical consistency between the 3D reconstruction, camera trajectory, and the stereo ground truth. It successfully transitioned from a rigid "at rest" state to highly deformed states without losing track. The sequential nature of the EKF allowed for accurate tracking of complex, non-linear deformations despite the underlying linear elastic model.
Computational efficiency is a major benefit; the primary overhead compared to standard EKF-SLAM is the inversion of a sparse stiffness matrix, making real-time application feasible for maps of approximately 100 points. However, current limitations include the requirement for an initial rigid phase for structure initialization and the need to predefined boundary conditions.
Conclusion
This work successfully demonstrates that embedding FEM-based physical priors into a monocular SLAM loop enables robust tracking of deforming surfaces. Future work will focus on full real-time implementation and testing on medical endoscope sequences using natural landmarks rather than artificial markers, moving towards more complex viscoelastic material models.
Summary updated