Introduction: The Challenge of Occlusions in 3D Scanning
3D scanning has become a cornerstone of Cultural Heritage (CH) documentation, yet achieving a perfect model remains a significant challenge. Scanners frequently miss sections of an object due to "occlusions"—tight folds, deep crevices, or angles inaccessible to the laser or light source.
The standard solution in computer graphics is "hole filling," which utilizes algorithms to bridge gaps based on surrounding geometry. However, for CH experts, this is problematic: these algorithms create "plausible" but ultimately synthetic data. If a statue has a missing finger or a specific decorative flourish within a hole, an interpolation algorithm will simply smooth it over. This research addresses the need for a low-cost, portable way to recover the actual geometry of these unsampled regions without requiring a full, expensive rescanning setup.
Methodology: Image-Guided Reconstruction
The authors propose an "image-guided" reconstruction pipeline that effectively transforms a digital camera and a simple projector (such as a laser diode) into a localized 3D scanner.
The pipeline consists of several key stages:
- Pattern Projection: A known pattern, such as a grid or square, is projected onto the area where the 3D model contains a hole.
- Image-to-Geometry Alignment: A photograph is taken, and the 2D image is aligned with the existing 3D model. This is currently a semi-automatic step requiring the identification of matching points.
- Pattern Extraction: The software analyzes the image to extract the projected pattern, using color channel filtering and erosion techniques to locate the center of the lines.
- Projector Calibration: Utilizing the Levenberg-Marquardt algorithm, the system estimates the exact position and orientation of the projector relative to the camera and the 3D model.
- Triangulation and Refinement: By knowing the spatial relationship between the camera, projector, and the pattern's shape on the surface, the system calculates 3D coordinates within the hole. These points are then used to create triangle strips that bridge the gap, followed by a remeshing and smoothing process to ensure a seamless blend with the original scan.
Key Insights and Results
The core philosophy of this approach is prioritizing "truth over plausibility." In Cultural Heritage, guessed data is often considered worse than no data. By using a projected pattern, the method moves from predicting a surface to measuring it.
The system also leverages redundancy to improve accuracy. By analyzing parts of the projected pattern that fall on known geometry outside the hole, the system calculates a "correction factor" to account for minor errors in projector position estimation. Furthermore, the method is hardware-agnostic, functioning effectively with either high-end digital projectors or inexpensive fixed-pattern laser diodes.
Quantitative and Qualitative Performance
The method was evaluated against standard techniques like Liepa (surface-oriented filling) and EarCut (simple triangulation) using both real-world CH objects and synthetic test cases.
- Quantitative: Using the Hausdorff distance to measure surface deviation, the image-guided method consistently showed lower Root Mean Square Error (RMSE). In one test case, the proposed method achieved an RMSE of 0.0079, significantly outperforming Liepa's 3.379.
- Qualitative: Visual analysis confirms that while standard methods smooth over missing details like clothing folds, the image-guided approach successfully recovers the actual physical ridges and valleys of the object.
Conclusion and Future Directions
While the current implementation requires some manual intervention for initial alignment and pattern identification, it represents a significant step toward high-fidelity CH preservation. Future work aims to fully automate the alignment process and integrate data from multiple camera angles to improve coverage. The ultimate goal is the development of a dedicated, low-cost "hole-filling" device for field researchers to use during initial data acquisition campaigns.
Summary updated