Multi-View Stereo 3D Edge Reconstruction

Andrea Bignoli, Andrea Romanoni, Matteo Matteucci

Abstract

This paper presents a novel method for the reconstruction of 3D edges in multi-view stereo scenarios. Previous research in the field typically relied on video sequences and limited the reconstruction process to either straight line-segments, or edge-points, i.e., 3D points that correspond to image edges. We instead propose a system, denoted as EdgeGraph3D, able to recover both straight and curved 3D edges from an unordered image sequence. A second contribution of this work is a graph-based representation for 2D edges that allows the identification of the most structurally significant edges detected in an image. We integrate EdgeGraph3D in a multi-view stereo reconstruction pipeline and analyze the benefits provided by 3D edges to the accuracy of the recovered surfaces. We evaluate the effectiveness of our approach on multiple datasets from two different collections in the multi-view stereo literature. Experimental results demonstrate the ability of EdgeGraph3D to work in presence of strong illumination changes and reflections, which are usually detrimental to the effectiveness of classical photometric reconstruction systems.

Introduction

Traditional Multi-View Stereo (MVS) pipelines typically rely on Structure from Motion (SfM) to generate sparse point clouds, followed by photometric matching to produce dense reconstructions. However, these methods often struggle with photometric sensitivity—where lighting changes and reflections create noise—and a lack of geometric salience, often failing to sharply define object boundaries. While previous attempts at 3D edge reconstruction existed, they were largely restricted to video sequences or straight-line segments. This paper introduces EdgeGraph3D, a system capable of recovering both straight and curved 3D edges from unordered image collections.

Methodology

The EdgeGraph3D pipeline integrates several stages to transform 2D image data into a structured 3D edge representation.

The EdgeGraph3D pipeline overview, from 2D edge-graph generation to final 3D edge reconstruction.

2D Edge-Graph Generation

The process begins by representing edges as a graph of pixels. To handle the "visual noise" inherent in raw edge detection, the system employs a "regular length" filtering technique. This prunes insignificant edges, retaining only structurally meaningful polylines that represent the true geometry of the scene.

Comparison of edge detection before and after the filtering process on the fountain-p11 dataset.

Correspondence and Reconstruction

To match these edges across views, the system identifies Potential Edge Correspondences (PEC) by leveraging existing 3D points from SfM. If 2D polylines in different images share proximity to the same 3D points, they are considered candidates. Potential Edge-Point Correspondences (PEPC) are then generated via epipolar geometry. The system validates these points across three views and recursively "walks" along the 2D edges to grow the 3D curve. Finally, a visibility refinement step filters outliers and optimizes the 3D edge positions.

Experimental Evaluation

The system was evaluated on the fountain-p11 (EPFL) and DTU datasets. These benchmarks provide high-resolution architectural details and complex geometries with challenging materials.

Key Results

  • Accuracy: On the fountain-p11 dataset, EdgeGraph3D achieved a 36% improvement in Mean Absolute Error (MAE) compared to standard OpenMVG-based meshes.
  • Curve Handling: Compared to Line3D++, which is limited to straight segments, EdgeGraph3D reduced mesh Root Mean Square Error (RMSE) by approximately 17%.
  • Robustness: The method proved particularly effective on "shiny" objects (e.g., metallic cans in DTU-098) where photometric methods often fail due to moving reflections.

Discussion and Conclusion

By using edges as structural anchors, EdgeGraph3D provides a more reliable "skeleton" for surface meshing than random point clouds. This representation is not only more accurate but also more compact. Future work includes integrating 3D edges directly into the Bundle Adjustment process to improve camera orientation robustness and utilizing constrained 3D Delaunay triangulation to ensure mesh faces align perfectly with reconstructed polylines.

Summary updated

Abstract

This paper presents a novel method for the reconstruction of 3D edges in multi-view stereo scenarios. Previous research in the field typically relied on video sequences and limited the reconstruction process to either straight line-segments, or edge-points, i.e., 3D points that correspond to image edges. We instead propose a system, denoted as EdgeGraph3D, able to recover both straight and curved 3D edges from an unordered image sequence. A second contribution of this work is a graph-based representation for 2D edges that allows the identification of the most structurally significant edges detected in an image. We integrate EdgeGraph3D in a multi-view stereo reconstruction pipeline and analyze the benefits provided by 3D edges to the accuracy of the recovered surfaces. We evaluate the effectiveness of our approach on multiple datasets from two different collections in the multi-view stereo literature. Experimental results demonstrate the ability of EdgeGraph3D to work in presence of strong illumination changes and reflections, which are usually detrimental to the effectiveness of classical photometric reconstruction systems.
BibTeX
@inproceedings{bignoli2018multi,
  author = {Andrea Bignoli and Andrea Romanoni and Matteo Matteucci},
  title = {{Multi-View Stereo 3D Edge Reconstruction}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2018},
  url = {https://openreview.net/forum?id=Q9fNtv4sBF},
}
Figure