Introduction
Disparity estimation is a cornerstone of 3D computer vision, enabling depth perception from stereo image pairs. While deep learning models, particularly 3D CNNs, have achieved state-of-the-art performance, they consistently struggle with over-smoothing at object boundaries. When these disparity maps are projected into 3D space, foreground objects often appear to have "long tails" of points stretching toward the background. This "bleeding" effect is not merely a visual artifact; it represents a structural failure that can compromise autonomous navigation and robotic mapping.
The Root Cause: Multimodality and Weighted Averaging
The researchers trace this over-smoothing back to the standard weighted average operation (or soft argmin) used in most modern architectures. At the boundary between a foreground object and the background, a pixel's receptive field often encompasses both depth levels. Consequently, the network predicts a multimodal probability distribution with two distinct peaks.
As shown in the figure above, when the network averages across the entire range, the resulting disparity value lands in the low-probability valley between the peaks. This intermediate value creates the characteristic blur at edges.
Proposed Solutions
To address this, the authors propose two primary modifications that require no additional parameters or complex semantic supervision:
- Single-Modal Weighted Average (SM): During inference, instead of averaging across the full disparity range, the algorithm identifies the highest probability peak. The weighted average is then calculated only within the local neighborhood of that peak, effectively ignoring "distractor" modes from the background or foreground.
- Cross-Entropy with Gaussian Supervision: Standard regression losses (like Smooth L1) do not sufficiently constrain the shape of the predicted distribution. The authors replace this with a Cross-Entropy loss that supervises the network to produce a target Gaussian distribution centered at the ground truth. This forces the model to learn sharper, unimodal-like distributions.
Experimental Results
The proposed method was evaluated on major benchmarks including SceneFlow, KITTI, and Middlebury. By applying these changes to existing models like PSMNet, the researchers observed a dramatic reduction in edge artifacts. On the SceneFlow dataset, the Soft Edge Error (SEE)—a metric designed to measure boundary sharpness—improved significantly, dropping from 9.40 to 4.17 for PSMNet.
The qualitative impact is most visible in 3D point cloud reconstructions. As seen in the comparison above, the "long tail" effect is substantially mitigated, resulting in cleaner object boundaries and more accurate spatial representations.
Conclusion
By identifying the mathematical source of over-smoothing, this work provides a simple yet powerful framework for improving stereo matching. The combination of Single-Modal inference and distribution-based training allows CNNs to maintain the benefits of global context while recovering the sharp local details necessary for reliable 3D vision.
Summary updated