Incremental Structure from Motion for Small-Scale Scenes Based on Auxiliary Calibration

Sixu Li, Jiatian Li, Tao Yang, A Xiaohui, Jiayin LiuSensors 2025 · DOI

TL;DR

This paper introduces a novel incremental Structure from Motion (SfM) method specifically designed for small-scale scenes, which often suffer from a lack of distinct features. By incorporating an auxiliary calibration plate and a feature enhancement strategy, the method significantly improves the accuracy and completeness of 3D reconstructions, achieving state-of-the-art re-projection errors.

What Problem Does This Paper Address?

Structure from Motion (SfM) is a powerful technique for reconstructing 3D scenes from a series of 2D images. However, a major hurdle, especially in small-scale environments like indoor spaces, is the scarcity of distinctive feature points. This "weak texture" problem leads to inaccurate camera pose estimation and incomplete 3D reconstructions. Existing SfM methods struggle to reliably identify and match enough features in such scenarios, leading to errors and instability.

Methods

The proposed method tackles the weak texture problem through a two-pronged approach:

  • Auxiliary Calibration Plate: A standard black-and-white checkerboard is used as an auxiliary calibration plate. This plate provides known geometric constraints (plane, orthogonality) that are crucial for robustly estimating camera poses, even when the main object has few features. The corner points of the checkerboard are used to:
  • Estimate initial camera poses.
  • Refine these poses by enforcing plane and line constraints derived from the checkerboard's geometry.
  • Filter out inaccurate feature matches by leveraging the precise calibration information.
  • Feature-Point Enhancement Algorithm: For regions with sparse features, the paper proposes a method to increase the number of reliable matches. This involves:
  • Randomly generating points in weak texture areas.
  • Using epipolar geometry constraints to find potential matches.
  • Further refining these matches using a "Simpson error" metric, which combines geometric and pose information.
  • Finally, selecting the best matches based on re-projection errors, leveraging the already estimated camera poses.

The overall pipeline involves initial camera pose estimation using the calibration plate, followed by an incremental image addition process where feature enhancement and matching are performed, and finally, a global bundle adjustment to refine the entire reconstruction.

Datasets

The paper evaluates its method on:

  • Three real-world experimental datasets: These datasets are not explicitly named or described in detail beyond their use in demonstrating the method's performance.
  • ETH3D dataset: Specifically, three indoor image sets from this dataset were used: lecture_room, living_room, and delivery_area. This dataset is known for its diverse environments and challenges.

No new benchmarks were introduced; rather, existing datasets were used to compare against established methods.

Key Insights and Ideas

  • Calibration Plates as Feature Augmentors: The core insight is that a simple calibration plate, even if not the primary object of interest, can act as a powerful "feature anchor" in SfM. Its known geometry provides strong constraints that compensate for the lack of features on the actual scene.
  • Hierarchical Constraint Application: The method effectively uses constraints in a hierarchical manner: first, the checkerboard's planar and orthogonal properties for initial pose estimation, then refined geometric constraints for filtering matches, and finally, re-projection error minimization for overall optimization.
  • Targeted Feature Enhancement: Instead of applying a generic feature enhancement to the entire image, the method intelligently focuses on weak texture regions, making it more efficient and effective.
  • Balancing Accuracy and Complexity: The proposed feature enhancement, while adding a step, aims to improve accuracy without drastically increasing computational complexity, making it suitable for incremental SfM.

Key Results

  • Improved Accuracy: The proposed method consistently achieved lower re-projection errors compared to competing methods. In three real-world experiments, average re-projection errors were 0.5245, 0.4151, and 0.4996 pixels. On the ETH3D dataset, the average re-projection error was 0.4151 pixels, outperforming methods from [19] and [20] by significant margins (0.2185 and 0.1945 pixels, respectively).
  • Denser Reconstructions: The method produced more complete and denser point clouds, offering better visualization.
  • Robust Feature Matching: The feature enhancement algorithm significantly increased the number of correctly matched feature points, especially in challenging, weakly textured scenes. For example, in one dataset, the number of correctly matched points increased by 486 and 225 compared to SIFT and AKAZE, respectively.
  • Competitive Runtime: While not the fastest, the method's running time was comparable to or slightly higher than traditional methods like SIFT, indicating that the added complexity did not lead to excessive computational overhead.

Limitations and Future Work

  • Reconstruction Beyond Calibration Scene: The authors acknowledge that due to the nature of the auxiliary calibration scene, certain parts of the 3D model that are not visible in conjunction with the calibration plate might not be reconstructed.
  • Future Work: The authors plan to focus on improving the reconstruction of these "beyond calibration scene" areas to enhance the overall reconstruction and visual quality. This suggests exploring methods to extend the reconstruction coverage or integrate information from views where the calibration plate is not present.

Summary updated

Abstract

Scarce feature points are a critical limitation affecting the accuracy and stability of incremental structure from motion (SfM) in small-scale scenes. In this paper, we propose an incremental SfM method for small-scale scenes, combined with an auxiliary calibration plate. This approach increases the number of feature points in sparse regions, and we randomly generate feature points within those areas. At the same time, we obtain a coarse matching set of feature points using pairwise polar geometric constraints. The positional results from the geometric constraints of the calibration plate are then used to filter out high-precision matching points, thereby improving the accuracy of the three-dimensional reconstruction. Experimental results demonstrate that the proposed method achieves superior reconstruction completeness and accuracy. In three real-world experiments, the average re-projection errors were 0.5245, 0.4151, and 0.4996 pixels, outperforming competing methods. This approach ensures robust pose estimation and facilitates precise 3D reconstructions.
BibTeX
@inproceedings{li2025incremental,
  author = {Sixu Li and Jiatian Li and Tao Yang and A Xiaohui and Jiayin Liu},
  title = {{Incremental Structure from Motion for Small-Scale Scenes Based on Auxiliary Calibration}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25020415},
}
Figure