Abstract
In industrial visual inspection, microscopes often suffer from a shallow depth of field, making it difficult to capture a single image where the entire object is in focus. This paper presents a deep learning-based method to generate "all-in-focus" (AIF) images from large focal stacks. The approach introduces two key innovations: an index map interpolation technique that reduces the number of images required for network input, and a training strategy using synthetic datasets enhanced with high-frequency textures. Experimental results demonstrate that these improvements lead to higher accuracy in AIF image generation for both synthetic and real-world industrial components.
Introduction
Microscopic inspection is a cornerstone of quality control for electronics and precision parts. However, the high magnification required for these tasks inherently results in a shallow depth of field. Consequently, only a thin slice of a 3D object—such as a circuit board or a bonding wire—appears sharp in a single capture. To overcome this, engineers typically acquire a "focal stack": a series of images taken at varying focal planes.
The challenge lies in efficiently merging these images into a single AIF image. Traditional "shape-from-focus" methods are often sensitive to image noise and lack robustness on textureless surfaces. While deep learning models offer a promising alternative, processing large focal stacks (often containing dozens of images) is computationally expensive and slow. This research addresses these bottlenecks by optimizing the data pipeline and enhancing the model's ability to recognize defocus cues.
Methodology: Efficient AIF Generation
The proposed system builds upon the DefocusNet architecture, introducing a novel workflow to handle large focal stacks efficiently.
Index Map Interpolation
A significant hurdle in deep learning for focal stacks is the memory and processing requirement for high-resolution image sequences. The researchers propose a "sparse input, dense output" strategy. Instead of feeding the entire stack of $N$ images into the network, only a subset of $n$ images (where $n \ll N$) is used.
The network predicts defocus maps for these $n$ images. From these, an initial index map is generated, identifying which of the $n$ images is most in focus at each pixel. To leverage the full detail of the original $N$-image stack, the system applies quadratic interpolation to these index values. This estimates the optimal focus index within the full range of $N$, allowing the final AIF image to be reconstructed using the sharpest possible pixels from the entire stack without the overhead of processing every image through the neural network.
Texture-Enhanced Synthetic Training
Neural networks trained on standard synthetic data often fail when encountering real-world materials that are smooth or have subtle textures. To improve the model's robustness, the authors generated synthetic training data using 3D models from the Thingi10K dataset but added a crucial step: the application of random procedural textures. By including high-frequency components like noise, waves, and checkerboards, the network is forced to learn the specific characteristics of defocus blur across a wide variety of visual patterns.
Results and Discussion
The method was evaluated using both synthetic benchmarks and real-world microscope data from industrial components, including LEDs and Ball Grid Arrays (BGA).
Quantitative Performance
On synthetic datasets, the proposed method achieved a Root Mean Square Error (RMSE) of 0.061 for depth estimation. This represents a substantial improvement over classical algorithms like Nayar’s method, which recorded an RMSE of 0.221. The interpolation technique proved highly effective; using just 5 input images, the model could accurately reconstruct focus information for a 19-image stack.
Qualitative Success
In practical tests on LED bonding wires—notoriously difficult targets due to their thinness and reflective properties—the proposed method successfully reconstructed sharp, continuous wires. Previous models often produced fragmented or blurred results in these regions. The addition of high-frequency textures during training was identified as a key factor in this improved performance, as it allowed the network to better distinguish between true focus and slight defocus blur on fine details.
Conclusion
This research demonstrates a practical and efficient path for integrating deep learning into high-speed industrial inspection. By combining sparse-input interpolation with texture-aware training, the proposed method generates high-quality all-in-focus images while minimizing computational costs. Future work will focus on optimizing the system for a broader range of materials and further reducing sensitivity to the specific step size of the focal stack.
Summary updated