Abstract
Depth from Focus (DFF) is a critical technique for 3D reconstruction, yet traditional spatial-domain convolutional neural networks (CNNs) often suffer from detail loss and blurring. We present the Frequency-Aware Deep Depth from Focus (FAD) network, a novel architecture that integrates global frequency-domain information via Fourier transforms into a deep learning pipeline. By combining local spatial features with global structural insights, FAD achieves state-of-the-art accuracy and superior detail preservation across synthetic and real-world datasets.
Introduction
Reconstructing 3D geometry from a sequence of images with varying focus—known as Depth from Focus (DFF)—is essential for high-resolution imaging and microscopy. However, modern DFF methods relying solely on spatial CNNs struggle with "near-sightedness." Because convolutions are inherently local operations, they often fail to maintain global structural consistency, leading to blurred edges and smoothed-over textures. This is particularly problematic in texture-less regions where local focus cues are weak and difficult to detect.
Methodology: The FAD Architecture
The Frequency-Aware Deep Depth from Focus (FAD) model overcomes these limitations through a hybrid spatial-frequency approach. The core innovation is the Time-Frequency Joint Module, which operates alongside standard spatial feature extraction within a U-Net-style backbone.
Key Components:
- 3D Convolutions: FAD employs 3D convolutions to process the input focal stack, capturing how focus evolves across the depth dimension (the temporal/focal axis).
- Clarity Area Identification (CAI): This specialized module targets subtle defocus variations, even in areas with weak texture.
- Frequency Domain Feature Extraction: By applying a Fast Fourier Transform (FFT) to latent features, the model gains a "mathematical bird's-eye view" of the scene. An adaptive weight matrix filters these frequencies to enhance relevant focus information before an inverse FFT (iFFT) returns the data to the spatial domain.
- Multi-Scale Weighted Loss: Training utilizes a pyramid-like loss function to ensure consistency across four different scales, from coarse shapes to fine details.
Experiments and Results
The FAD model was rigorously tested on a diverse set of benchmarks, including synthetic datasets (SLFD, Pov-Ray, 4D Light Field) and real-world datasets (NYU Depth V2, DDFF 12-Scene). It was also applied to microscopic samples through a pseudo-labeling fine-tuning process.
FAD consistently outperforms existing state-of-the-art (SOTA) methods like DfFW and AiF. Quantitatively, on the SLFD dataset, FAD achieved a Mean Squared Error (MSE) of 0.3460, a significant improvement over the previous best of 0.8939. Qualitatively, the model excels at reconstructing complex geometries—such as thin structures and surface scratches—that traditional models typically smooth over. Despite the added complexity of FFTs, the model remains efficient with an inference time of approximately 0.12 seconds per stack.
Conclusion
The FAD network demonstrates that integrating frequency-domain information is a highly effective strategy for high-fidelity 3D reconstruction. By balancing local spatial features with global frequency insights, the model sets a new standard for Depth from Focus. Future work will explore multi-task learning to develop a universal DFF model capable of zero-shot generalization across diverse optical systems.
Summary updated