Traditional stereo matching operates on an elegant premise: two identical cameras, separated by a horizontal baseline, capture the same scene with the same resolution, exposure, and optics. Under this symmetric assumption, finding depth is a matter of searching along horizontal epipolar lines for pixels or patches with matching intensities.
However, modern consumer hardware—especially smartphones—breaks this assumption entirely. To minimize physical footprint and manufacturing costs, device makers often pair a high-quality, high-resolution primary sensor with a lower-specification, lower-resolution secondary sensor, as noted by Liu et al.. This design creates a visual imbalance or asymmetric degradation, where one view is inherently sharper and more detailed, while the other is blurrier, noisier, or lower in resolution.
Why Asymmetry Breaks Traditional Matching
When algorithms designed for symmetric inputs encounter unbalanced stereo pairs, they suffer severe performance degradation. The core issue lies in the breakdown of photometric and structural consistency.
Consider an extreme scenario with a 12x resolution difference between the left and right sensors, a challenge explored by Tosi et al.. A sharp, high-frequency edge in the primary image corresponds to a blurry, aliased patch in the secondary image. Standard cost volumes, which compute similarity by comparing localized pixel patches, fail because the underlying data distributions and spatial frequencies differ drastically between the two views. As highlighted by Zama Ramirez et al., this mismatch introduces severe aliasing and matching ambiguities that standard deep learning architectures cannot natively resolve.
Bridging the Resolution Gap via Continuous Coordinates
To find correspondences across disparate sensor scales, recent architectures shift away from rigid, discrete pixel grids. Instead, they map features into a shared, continuous representation.
For instance, the framework proposed by Tosi et al. extracts features from the unbalanced sensor pairs and queries them within a common high-resolution coordinate space. By treating the disparity field as a continuous function rather than a discrete grid, the network can refine depth maps to arbitrary resolutions. This allows the system to preserve the fine-grained spatial details of the high-resolution reference image while successfully establishing correspondences with a significantly lower-resolution target image, a mechanism crucial for heterogeneous resolution stereo matching Zama Ramirez et al..
[High-Res Left Image] --> [Feature Extraction] ---\
--> [Continuous Coordinate Space] --> [Arbitrary Resolution Disparity]
[Low-Res Right Image] --> [Feature Extraction] ---/
Learning Invariance to Asymmetric Degradation
Resolution is not the only source of asymmetry; real-world lenses also suffer from independent variations in focus, exposure, and noise. If a network is trained only on clean, perfectly symmetric synthetic datasets, it fails when deployed on real hardware where one lens is inherently more distorted or blurrier than the other Li et al..
To force networks to learn robust, degradation-invariant features, the authors of recent stereo super-resolution frameworks have introduced independent, asymmetric degradation pipelines during training. As demonstrated by Zhou et al., by deliberately applying different blur kernels, noise levels, and compression artifacts to the left and right channels independently, models learn to look past surface-level discrepancies. They focus instead on the underlying structural geometry shared between the views.
When explicit ground-truth depth labels are unavailable, unsupervised approaches must find alternative ways to compute similarity without relying on direct pixel comparisons. Song et al. address this by introducing Spatially-Adaptive Self-Similarity (SASS). Instead of comparing raw pixel intensities across the asymmetric views, their framework adaptively learns unique sampling patterns for every pixel. By comparing how a pixel relates to its own neighborhood across both views, the network establishes accurate correspondences even when the absolute noise profiles or resolutions are completely mismatched.
Extreme Asymmetry: Cross-Modal Correspondence
The ultimate extension of asymmetric stereo matching involves pairing entirely different sensor modalities. Rather than matching two optical RGB images, some modern depth-sensing systems pair a standard RGB camera with an infrared (IR) camera that captures projected speckle patterns Xu et al..
This setup introduces extreme appearance discrepancies: one image contains rich color and lighting information, while the other contains monochrome, active illumination patterns. To resolve this, deep neural networks are trained to perform cross-modal feature extraction. Instead of matching raw appearance, the network projects both the RGB and IR images into a shared embedding space where the geometric structures of the scene are aligned. This proves that modern deep architectures, when properly constrained, can overcome severe visual asymmetry to produce highly reliable disparity maps from entirely different spectral domains.
Key Foundational Work
Neural Disparity Refinement for Arbitrary Resolution Stereo (2021) — contributed 3 concept elements to this direction.
Visually Imbalanced Stereo Matching (2020) — contributed 1 concept elements to this direction.