DualFocus: Depth from Focus with Spatio-Focal Dual Variational Constraints

Sungmin Woo, Sangyoun Lee

Abstract

Depth-from-Focus (DFF) enables precise depth estimation by analyzing focus cues across a stack of images captured at varying focal lengths. While recent learning-based approaches have advanced this field, they often struggle in complex scenes with fine textures or abrupt depth changes, where focus cues may become ambiguous or misleading. We present DualFocus, a novel DFF framework that leverages the focal stack's unique gradient patterns induced by focus variation, jointly modeling focus changes over spatial and focal dimensions. Our approach introduces a variational formulation with dual constraints tailored to DFF: spatial constraints exploit gradient pattern changes across focus levels to distinguish true depth edges from texture artifacts, while focal constraints enforce unimodal, monotonic focus probabilities aligned with physical focus behavior. These inductive biases improve robustness and accuracy in challenging regions. Comprehensive experiments on four public datasets demonstrate that DualFocus consistently outperforms state-of-the-art methods in both depth accuracy and perceptual quality.

Introduction

Estimating depth is a cornerstone of computer vision, yet different methods present distinct trade-offs. Monocular depth estimation often suffers from scale ambiguity, whereas Depth-from-Focus (DFF) utilizes a "focal stack"—a series of images captured at different focal lengths—to resolve absolute depth. The underlying physical principle is that a scene point is sharpest when the camera's focal plane aligns with its actual depth.

However, contemporary deep-learning DFF models frequently treat focal stacks as standard video or RGB data, neglecting the fundamental physics of light. This oversight leads to noisy depth maps, particularly in regions with fine textures, reflections, or abrupt depth transitions where focus signals are ambiguous. DualFocus addresses these limitations by re-integrating physical and geometric constraints into the learning framework.

Methodology

DualFocus introduces a novel architecture guided by two specific "variational constraints" that enforce physically plausible solutions during the depth estimation process.

The DualFocus architecture overview, illustrating the flow from focal stack input to final depth fusion through spatio-focal constraints.

Spatial Variational Constraints

Rather than directly regressing depth, the model predicts spatial gradients. It employs a least-squares optimization to project these gradients onto a space of "integrable" surfaces. This geometric "skeleton" forces the model to distinguish between true depth edges—which exhibit predictable changes across focus levels—and flat textures that might appear sharp but do not behave like physical depth boundaries.

Focal Variational Constraints

In physical optics, as a lens sweeps through focus, a point's sharpness follows a unimodal and monotonic curve: it sharpens to a single peak and then blurs. DualFocus enforces this behavior via a focal variational loss ($L_{fv}$) that penalizes predicted focus probabilities that deviate from this smooth, single-peak trend. This constraint significantly stabilizes depth estimation in low-texture areas, such as plain walls, where focus peaks are otherwise difficult to identify.

3D-Convolutional Fusion

These physics-aware features are processed by a 3D-convolutional decoder. By reasoning across both spatial ($x, y$) and focal ($z$) dimensions, the network produces a high-resolution depth map that respects both local image features and global focal consistency.

Experimental Evaluation

The framework was evaluated across four diverse benchmarks to test precision and generalization:

  • NYU Depth v2: Used for testing high-precision depth estimation on synthetic focal stacks.
  • FoD500: A synthetic dataset tailored for defocus and focus tasks.
  • DDFF 12-Scene: A real-world dataset providing complex, "messy" focus cues.
  • ARKitScenes: Utilized for zero-shot evaluation to demonstrate the model's ability to generalize to unseen environments without additional training.

Results and Discussion

DualFocus consistently outperforms existing state-of-the-art (SOTA) methods, such as DFV and HybridDepth, across all metrics. On the NYU Depth v2 dataset, the model achieved a 20.2% reduction in Root Mean Squared Error (RMSE) compared to previous benchmarks.

Qualitative comparison of depth maps produced by HybridDepth, DFV, and DualFocus against Ground Truth.

Beyond quantitative metrics, DualFocus provides superior perceptual quality. As shown in the qualitative comparisons, the model produces significantly sharper edges and fewer artifacts around complex geometries like furniture legs and thin structures. In zero-shot tests on ARKitScenes, DualFocus outperformed models with significantly higher parameter counts, validating that its physics-based inductive biases are more effective than raw computational scale.

Conclusion and Future Work

DualFocus demonstrates that integrating physical principles into deep learning architectures can overcome the limitations of purely data-driven DFF models. While the system significantly improves robustness, challenges remain in perfectly uniform, textureless regions where focus cues are absent. Future research will explore modeling asymmetric lens aberrations and improving robustness to camera motion during the focal sweep to further enhance real-world applicability.

Summary updated

Abstract

Depth-from-Focus (DFF) enables precise depth estimation by analyzing focus cues across a stack of images captured at varying focal lengths. While recent learning-based approaches have advanced this field, they often struggle in complex scenes with fine textures or abrupt depth changes, where focus cues may become ambiguous or misleading. We present DualFocus, a novel DFF framework that leverages the focal stack's unique gradient patterns induced by focus variation, jointly modeling focus changes over spatial and focal dimensions. Our approach introduces a variational formulation with dual constraints tailored to DFF: spatial constraints exploit gradient pattern changes across focus levels to distinguish true depth edges from texture artifacts, while focal constraints enforce unimodal, monotonic focus probabilities aligned with physical focus behavior. These inductive biases improve robustness and accuracy in challenging regions. Comprehensive experiments on four public datasets demonstrate that DualFocus consistently outperforms state-of-the-art methods in both depth accuracy and perceptual quality.
BibTeX
@inproceedings{woo2025dualfocus,
  author = {Sungmin Woo and Sangyoun Lee},
  title = {{DualFocus: Depth from Focus with Spatio-Focal Dual Variational Constraints}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=OZUl49U6p6},
}
Figure