In computer vision, predicting the distance of objects from a single image—monocular depth estimation—is famously "ill-posed." Because a 2D image is a projection of a 3D world, a single pixel could technically represent an object at any distance. Traditionally, researchers treated this as a regression problem: the model outputs a single continuous numerical value for every pixel.
However, a growing body of research suggests that regression might be the wrong tool for the job. Instead, the field is shifting toward treating depth as a discrete classification task. By dividing the continuous range of possible distances into a set of "bins" or "buckets," models can achieve sharper boundaries, more stable training, and a better grasp of scene geometry.
The Regression Trap
When a neural network performs regression, it tries to minimize the distance between its prediction and the ground truth (usually via L1 or L2 loss). While intuitive, this approach has two major drawbacks in depth estimation:
- Blurry Edges: Regression losses tend to "average out" uncertainty. If a model is unsure whether a pixel belongs to a foreground pole or a background wall, it might predict a value halfway between them, resulting in a "halo" or blurry edge.
- Optimization Difficulty: As Yuanzhouhan Cao and Zifeng Wu argued in Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks, identifying which depth range an object falls into is computationally easier for a deep network than predicting an exact numerical value. Regression landscapes are often riddled with local minima that classification avoids.
Binning the World
The core mechanism of this shift is discretization. Instead of asking "How many meters away is this?", the model asks "Which of these 100 depth intervals does this pixel belong to?"
This discretization can take several forms. In Pulling Things out of Perspective, Lubor Ladicky and colleagues simplified the problem by training classifiers to recognize objects at a single "canonical" depth. More recently, models like SQLdepth, developed by Youhong Wang and Hao Xu, treat depth as a classification problem over discrete bins in latent space, allowing for the recovery of much finer structural details SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimation.
To maintain the precision of a continuous output while using the stability of classification, many researchers use the soft-argmax operation. As explored by Adrian Johnston and Gustavo Carneiro, the model predicts a probability distribution across all bins. The final depth is the weighted average of these bins. This keeps the architecture end-to-end differentiable while preventing the "blurring" effect of standard regression Self-Supervised Monocular Trained Depth Estimation Using Self-Attention and Discrete Disparity Volume.
Why Classification Works Better
The move to classification provides three distinct advantages:
1. Sharper Geometric Boundaries In 3D reconstruction, surface transitions are often abrupt. Kamal Kumar Gupta and Abhinav Shrivastava found that discretizing depth into categorical levels allows models to capture sharp geometric boundaries that regression typically smooths out Improved Modeling of 3D Shapes with Multi-view Depth Maps. This is particularly useful in tasks like face anti-spoofing, where Dingsheng Luo and Xihong Wu used classification to distinguish the non-linear boundaries between real skin and spoofing materials Reinforcing Face Anti-Spoofing with Multi-Scale Modality.
2. Handling Depth Ambiguity Monocular images are inherently ambiguous. A classification approach allows a model to express this uncertainty. In pixelSplat, David Charatan and Vincent Sitzmann model depth as a categorical distribution. Sampling from this distribution during training prevents the model from getting stuck in local minima—a common failure when trying to optimize the placement of 3D primitives via regression.
3. Efficiency and Scalability Classification can be more memory-efficient. Zhenxing Mi and Dan Xu demonstrated that using discrete bins for iterative refinement reduces the search space in Multi-View Stereo (MVS), making it possible to reconstruct scenes without the massive memory overhead of traditional 3D cost volumes Generalized Binary Search Network for Highly-Efficient Multi-View Stereo.
Expanding the Horizon
This paradigm shift is not limited to simple depth maps. It is being applied to:
- Human Pose Estimation: Jun-Hee Kim and Seong-Whan Lee decomposed 3D depth into a binary classification task to resolve the ambiguity of whether a limb is reaching forward or backward Toward Approaches to Scalability in 3D Human Pose Estimation.
- LiDAR Compression: Sukai Wang and Ming Liu treated point cloud compression as a classification-based entropy modeling problem, achieving lossless compression by predicting the probability distribution of occupancy bins Point Cloud Compression with Range Image-Based Entropy Model for Autonomous Driving.
- Real-time Rendering: Xinyu Gao and Xiaogang Jin used multi-level classification across spatial bins to speed up ray-surface intersections in NeRF compositions by 30–40x A General Implicit Framework for Fast NeRF Composition and Rendering.
By reframing a continuous physical property (distance) as a discrete categorical choice, these researchers have found a way to make deep networks more robust, precise, and efficient at understanding the third dimension.