EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups

Zaiyan Yang, Jieji Ren, Xiangyi Wang, Ru Li, Xu Cao, Heng Guo, Zhanyu Ma, Boxin Shi

Abstract

Recent advancements in neural surface reconstruction have significantly enhanced 3D reconstruction. However, current real-world datasets mainly focus on benchmarking multiview stereo (MVS) based on RGB inputs. Multiview photometric stereo (MVPS) and multiview shape from polarization (MVSfP), though indispensable for high-fidelity surface reconstruction and sparse inputs, have not been quantitatively assessed together with MVS. To determine the working range of different MVX (MVS, MVSfP, and MVPS) techniques, we propose EvalMVX, a real-world dataset containing 25 objects, each captured with a polarized camera under 20 varying views and 17 light conditions including OLAT and natural illumination, leading to 8,500 images. Each object includes aligned ground-truth 3D mesh, facilitating quantitative benchmarking of MVX methods simultaneously. Based on our EvalMVX, we evaluate 13 MVX methods published in recent years, record the best-performing methods, and identify open problems under diverse geometric details and reflectance types.

Introduction

In the world of computer vision, 3D reconstruction has seen a "Cambrian explosion" since the arrival of Neural Radiance Fields (NeRF). However, researchers typically work in silos based on their input data: Multiview Stereo (MVS) uses standard RGB images; Multiview Photometric Stereo (MVPS) uses varying light directions to capture fine surface details; and Multiview Shape from Polarization (MVSfP) uses polarized light to resolve ambiguities on shiny or reflective surfaces.

Until now, there was no single dataset that allowed researchers to test all three approaches on the same objects under the same conditions. Existing datasets like DTU focus almost exclusively on MVS, while others lack the high-quality ground-truth (GT) 3D scans needed for quantitative comparison. EvalMVX addresses this gap by providing a unified benchmark to establish a "working range" for these techniques.

Methodology and Capture Setup

The authors built a custom capture rig featuring a Lucid Triton RGB polarization camera and a 3D-printed ring of 16 high-power LEDs. For each of the 25 objects, they captured 20 viewpoints. At each viewpoint, they took one image under ambient light and 16 "One-Light-At-A-Time" (OLAT) images.

The EvalMVX capture setup and multi-modal data acquisition.

To ensure the benchmark was fair, they developed a sophisticated GT shape alignment pipeline. They scanned each object with a professional 3D scanner and used an "inverse silhouette rendering" technique to perfectly align the 3D mesh with the camera views. This allows for pixel-perfect error measurement. Additionally, the researchers introduced a technical improvement to a leading MVPS method, SuperNormal, by incorporating "integrated depth" as a geometric constraint to smooth out discontinuities in reconstructed meshes.

The EvalMVX Dataset

The primary contribution is the EvalMVX dataset, which includes:

  • 25 objects: Ranging from simple diffuse surfaces (a plaster face) to complex metallic ones (a brass bell) and translucent materials (a rubber duck).
  • 8,500 images total: Captured with polarization data, allowing for the extraction of both standard RGB and polarized Stokes vectors.
  • Unified Benchmarking: Unlike previous datasets, EvalMVX is the first to support MVS, MVPS, and MVSfP benchmarking simultaneously with aligned GT meshes.

Benchmarking Results and Insights

The authors evaluated 13 recent methods using Chamfer Distance (CD). A key finding is that there is no "one size fits all" solution; the best reconstruction method depends heavily on the material and geometric complexity.

Best performing methods categorized by object complexity and reflectance.

Key Findings:

  • Top Performer: SuperNormal (MVPS) achieved the lowest median CD (0.314 mm) across the entire dataset, generally proving most accurate for complex geometries.
  • Material Specifics: NeRO (an MVS method) excels at highly reflective metallic surfaces, while WildLight achieved a CD of 0.200 mm on metallic objects like the "Bell," significantly beating standard MVS approaches.
  • The Power of Light: MVPS methods generally outperform MVS because varying light directions provide richer cues about surface orientation than static images.
  • Polarization's Potential: While MVSfP (like the PISR method) is theoretically robust for translucent materials, it is currently hampered by sensor noise in snapshot polarization cameras.

Conclusion

EvalMVX reveals that while neural reconstruction has advanced, significant bottlenecks remain in the reliability of geometric priors. Surface normals estimated from learning-based photometric stereo or polarization are still prone to errors due to "domain gaps" between synthetic training data and real-world physics. Future work will likely focus on using multiview consistency to "self-correct" these noisy priors and expanding the dataset to include depth and infrared (IR) camera data.

Summary updated

Abstract

Recent advancements in neural surface reconstruction have significantly enhanced 3D reconstruction. However, current real world datasets mainly focus on benchmarking multiview stereo (MVS) based on RGB inputs. Multiview photometric stereo (MVPS) and multiview shape from polarization (MVSfP), though indispensable on high-fidelity surface reconstruction and sparse inputs, have not been quantitatively assessed together with MVS. To determine the working range of different MVX (MVS, MVSfP, and MVPS) techniques, we propose EvalMVX, a real-world dataset containing $25$ objects, each captured with a polarized camera under $20$ varying views and $17$ light conditions including OLAT and natural illumination, leading to $8,500$ images. Each object includes aligned ground-truth 3D mesh, facilitating quantitative benchmarking of MVX methods simultaneously. Based on our EvalMVX, we evaluate $13$ MVX methods published in recent years, record the best-performing methods, and identify open problems under diverse geometric details and reflectance types. We hope EvalMVX and the benchmarking results can inspire future research on multiview 3D reconstruction.
BibTeX
@misc{yang2026evalmvx,
  author = {Zaiyan Yang and Jieji Ren and Xiangyi Wang and Ru Li and Xu Cao and Heng Guo and Zhanyu Ma and Boxin Shi},
  title = {{EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups}},
  year = {2026},
  eprint = {2602.24065},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.24065},
}
Figure