Guided Depth Map Super Resolution

30 concepts30 papers1 proposal

Modern imaging pipelines frequently combine multiple sensors to capture different modalities of a scene. However, these hardware configurations suffer from an inherent asymmetry: while standard RGB cameras are inexpensive and capture high-resolution (HR) visual information with millions of pixels, active depth sensors like LiDAR, Time-of-Flight (ToF) cameras, thermal sensors, and radio-frequency devices are physically constrained to produce low-resolution (LR), noisy, or sparse data.

This discrepancy creates a fundamental challenge for downstream tasks like 3D reconstruction, autonomous driving, and augmented reality, which require pixel-perfect alignment between visual textures and geometric depth maps. Simply interpolating a low-resolution depth map using standard bilinear or bicubic methods blurs geometric edges and amplifies sensor noise. To bridge this gap, researchers developed Guided Depth Map Super-Resolution (GDSR), a family of techniques that leverage the sharp structural details of a high-resolution companion image to intelligently upscale a low-resolution target map.

Why Guidance Works: Structural Co-occurrence

The foundational insight behind guided upsampling is that changes in a scene's geometry (depth discontinuities) almost always align with changes in its visual appearance (color or texture edges). If a camera sees a sharp boundary between a foreground object and the background, the depth sensor will likely experience a sharp step-change in distance at that exact location.

Early approaches formalized this intuition using probabilistic graphical models. In the seminal work An Application of Markov Random Fields to Range Sensing, James Diebel et al. utilized a Markov Random Field (MRF) where the prior distribution enforces depth smoothness but allows sharp depth discontinuities precisely where high-resolution color edges exist. By treating the high-resolution image as a structural prior, the authors achieved an order-of-magnitude increase in depth resolution while simultaneously suppressing sensor noise.

As the field progressed, researchers sought more computationally efficient ways to measure and transfer these structural boundaries. Instead of standard Euclidean distance filtering, Ming-Yu Liu et al. introduced Joint Geodesic Upsampling of Depth Images, which measures distances along the image manifold. By computing geodesic paths on the high-resolution guidance image, the method ensures that depth values are only propagated within visually homogeneous regions, preventing blurring across object boundaries. Similarly, Meiqin Liu et al. leveraged the self-referential properties of spatial data via fractal transforms in Resolution-independent Up-sampling for Depth Map Using Fractal Transforms, enabling upsampling to arbitrary target resolutions without losing edge sharpness.

Mitigating Artifacts: Texture Copying and Depth Bleeding

While transferring high-resolution structures from RGB to depth is highly effective, it introduces a critical vulnerability: the cross-modality gap. Visual edges do not always correspond to geometric edges. For instance, a checkerboard pattern painted on a perfectly flat wall creates strong visual gradients but zero depth variation. A naive guided upsampling algorithm will mistakenly inject these visual textures into the depth map—an artifact known as texture copying. Conversely, when a depth boundary is slightly misaligned with a visual edge, depth values can spill over incorrectly, causing depth bleeding.

To combat these artifacts, modern architectures focus on uncertainty estimation and domain-aware feature transmission. In Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution, Wuxuan Shi et al. proposed an iterative up-and-down sampling pipeline that explicitly models the cross-modality gap. By estimating the uncertainty of the guidance features, their framework prevents irrelevant RGB textures from corrupting flat geometric surfaces.

Other methods address this by operating in alternative feature spaces or using advanced diffusion mechanisms. The authors of SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-resolution explicitly extract high-frequency guidance cues in both the gradient and frequency domains, isolating sharp structural boundaries from distracting surface textures. In Guided Depth Super-Resolution by Deep Anisotropic Diffusion, Nando Metzger et al. combined deep learning with classical anisotropic diffusion, allowing the network to dynamically adjust the direction and strength of depth propagation based on contextual reasoning. To eliminate depth bleeding entirely, Athanasios Tragakis et al. introduced IGAF: Incremental Guided Attention Fusion for Depth Super-Resolution, which employs cross-modal attention to iteratively refine how features are fused across scales.

Expanding Beyond Pure Depth Upsampling

The principles of resolving sensor resolution disparity extend far beyond upscaling static depth maps. The interplay between high-resolution visual data and low-resolution geometric or physical sensors has unlocked novel capabilities across computer vision:

By formalizing how high-resolution structural context can safely guide low-resolution physical measurements, these diverse approaches turn a fundamental hardware limitation into a cooperative multi-sensor advantage.

Key Foundational Work

An Application of Markov Random Fields to Range Sensing (2005) — contributed 1 concept elements to this direction.
Registration of Range Image Using Texture of High-Resolution Color Images (2002) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_sensor_resolution_disparity_ced03d24,
  title = {sensor resolution disparity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sensor-resolution-disparity/rdr_ced03d2439fc47d0bca8bf63133d1299},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_sensor_resolution_disparity_ced03d24_v1,
  title = {sensor resolution disparity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sensor-resolution-disparity/rdr_ced03d2439fc47d0bca8bf63133d1299/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake