A single camera image cannot say how far away an object is. Every point along the line of sight through a pixel lands on that same pixel, so a small car nearby and a large car far away can look identical. Monocular 3D detectors, which place 3D boxes around objects from one image, cope with this by learning statistical cues such as apparent size and height in the image. Those cues break under occlusion, truncation at the image border, and changes in the camera. Classical geometric vision handled a similar problem differently. Equation solvers produce many mathematically valid but physically impossible answers, called "ghost" solutions, and these are removed by applying physical rules: a point must lie in front of the camera, and a visible surface cannot be hidden. This note argues that the same rules, written as inequalities rather than equalities, can turn the infinite depth ambiguity along a ray into a bounded interval for each object. A learned depth distribution can then be cut down to that interval during both training and inference.
Put simply, before trusting a network's depth guess for an object, we compute the range of depths at which a box of the predicted size and orientation is physically consistent with what the image shows. That means its visible edges, the parts cut off by the image border, the parts hidden by other objects, and every box corner lying in front of the camera. We then let the network choose only within that range. The bet is that this helps most where current methods fail worst: objects that are occluded, truncated, or seen through an unfamiliar camera.
Core Claim
The positive-depth rule, also called chirality or cheirality ("every visible 3D point has positive depth"), is almost useless by itself for driving scenes. Objects are nearly always far in front of the camera, and clamping depth to be positive costs nothing. Constraints that actually reduce the range of possible depths come from visibility as a whole. 3D Bounding Box Estimation Using Deep Learning and Geometry solves for 3D translation by requiring the projected box to touch all four sides of the 2D detection, which is a set of equalities. That system is wrong exactly when an object is truncated or occluded, because then the true box extends past the observed edge. Our central claim is that replacing each such equality with the correct inequality gives a feasible depth interval $F_i = [z^{\min}_i, z^{\max}_i]$ for each object $i$. The inequalities are an "extends at least this far" condition on occluded and truncated sides, equality only on sides that are actually visible, a requirement that all eight box corners have positive depth and that the camera is outside the box, and pairwise ordering constraints between objects whose 2D regions overlap. The width of $F_i$ is a geometric measure of how well that object's depth is determined. Restricting the detector's depth distribution to $F_i$ should reduce gross depth errors on the KITTI Hard split and under camera shifts more than any training-time remedy does.
Concretely, if $p_i(z)$ is the categorical depth distribution a detector such as MonoDETR predicts for object $i$ over depth bins $z$, we replace it with
$$ \tilde p_i(z) \propto p_i(z)\, \exp\!\big(-d(z, F_i)^2 / 2\sigma^2\big), $$
where $d(z, F_i)$ is the distance from bin $z$ to the feasible interval (zero inside it) and $\sigma$ absorbs pixel noise in the 2D detections. The soft margin matters. The minimal-solver literature reports that strictly enforcing constraints under noisy correspondences causes solvers to fail outright, and a hard cut on a noisy 2D box would cause the same failure here.
Research Program
The geometric core follows a design choice taken from Lambda Twist, which reformulates three-point camera pose estimation to solve directly for point depths. That lets it reject negative-depth roots inside the solver, and it reports 32 failures versus 4,210 and 5,027 for competing solvers over $10^7$ near-degenerate samples. We likewise parameterize each object along its center ray by depth alone. The detector already predicts dimensions and orientation, which 3D Bounding Box Estimation Using Deep Learning and Geometry shows are the low-variance, well-cued quantities. Given those, each projected box edge is a known function of depth along the ray. Each visibility condition is therefore a one-dimensional inequality in $z$, and intersecting the intervals they define costs microseconds per object.
Three kinds of side label determine which inequality applies. A truncated side, where the 2D box touches the image border, is labeled at no cost from the border itself. A visible side gets equality within margin $\sigma$. A side abutting another detection is occluded or occluding, and which of the two holds is unknown. We treat that as a two-way choice: both orderings generate an interval, and the pair of objects keeps the ordering whose intervals are jointly nonempty and best supported by $p_i$ and $p_j$. This is a small version of the "pick & solve" idea from Learning to Solve Hard Minimal Problems, in which a learned model selects the physically consistent branch instead of enumerating all algebraic roots. Here the branches are occlusion orderings rather than homotopy anchors, and the selector is the detector's own depth distribution. Scene-level ordering constraints are propagated by repeatedly tightening intervals over the overlap graph, which converges in a few passes on driving scenes.
We apply the interval in two places. At inference it reweights the depth distribution as above. During training, a loss on probability mass outside $F_i$, with $F_i$ computed from ground-truth dimensions and side labels, teaches the depth head to place mass inside the region consistent with visibility. This is the inference-time integration that Exploring Geometric Consistency for Monocular 3D Object Detection leaves open: Geo-Aug enforces perspective consistency only through data augmentation and offers no guarantee at test time. We also build the same interval into the depth hypotheses of a cost-volume detector. MonoJSG builds its semantic-geometric cost volume over fixed heuristic depth ranges. Sampling hypotheses only inside $F_i$ is the single-view analogue of how Segmenting Epipolar Line uses chirality to keep only one segment of the epipolar line, which roughly halves the search on real scenes.
Several papers motivate the program by documenting the failure it targets. Geo-Aug shows that detectors break when object scale or vertical position is perturbed. Disentangling Monocular 3D Object Detection shows that depth errors push networks toward physically wrong sizes. MAGneT-3D reports degradation on truncated objects and new camera intrinsics. One cited paper constrains the design. RayFusion removes ghost detections by intersecting rays from several vehicles, which tells us that single-view constraints can at best bound depth and cannot pin it down. Our claim is accordingly about interval width and gross-error reduction, not about recovering exact depth.
Baselines
The naive reference point is MonoDETR (Zhang et al., ICCV version) run unchanged, with depth clamped to be positive. That clamp is the "chirality alone" control, and we expect it to change almost nothing. MonoDETR is also the host detector, because it already predicts depth bins with linearly increasing width and is supervised only by object-level depth, so it needs no LiDAR. The file reports only relative gains for MonoDETR on the KITTI test set (+2.53% Easy and +1.08% Moderate $AP_{3D}$ over the previous leaders), and no absolute number for it was found on Lacuna. We will therefore reproduce it with its released configuration, and every variant will share its backbone, schedule, data, and tuning budget.
The strongest geometry-based competitor with a reported absolute number is MonoJSG, at 16.14% AP on KITTI Car Moderate (MonoJSG). We will run it with its fixed depth range and with the interval-restricted range, at equal numbers of depth hypotheses. CaDDN (Reading et al.) is a reference for depth distributions trained with LiDAR supervision. Older reference points are MonoDIS at 7.60% Car Moderate $AP_{R40}$ on the test set (Disentangling) and Geo-Aug applied to MonoDETR (Geometric Consistency). For Geo-Aug, the reported cross-domain improvement from 10.23 to 19.40 nuScenes mAP when training on KITTI is the target to beat for transfer.
Two ablations isolate the key ingredient. The first applies the original tight-fit equality constraints of Mousavian et al. as the interval. This tests whether the inequality treatment of truncated and occluded sides is what helps. The second drops the pairwise ordering constraints, which measures the value of reasoning about the whole scene. Beating the baselines means a statistically consistent gain over the reproduced MonoDETR on KITTI Hard $AP_{R40}$ and on the subset with occluded or truncated objects, with no loss on Easy, across three seeds.
Why not just…?
Why not just rely on the learned depth distribution, which already hedges across bins? MonoDETR's distribution is still trained on the spatial statistics of the training set. Geo-Aug and MAGneT-3D report that such detectors fail exactly where those statistics shift. A hedged distribution can still put most of its mass on a depth at which the box would extend through the image border or sit in front of an object that occludes it. The plain MonoDETR baseline, evaluated on the occluded and truncated subsets, measures how often this happens. We will report the fraction of its predictions that fall outside $F_i$ before claiming any improvement.
Why not just use Geo-Aug and keep geometry out of the model? Augmentation teaches tendencies, not limits, and its authors apply consistency only during training. It does not address occlusion ordering at all. This alternative is a real threat on clean, unoccluded cars. We therefore test Geo-Aug alone, our method alone, and the two combined, and we expect them to be complementary rather than redundant.
Why not just generate many candidate boxes and discard the physically inconsistent ones afterward? This is the most serious alternative, because the constraint set is the same and only the place it is applied differs. We will run it directly: sample $K$ depths from $p_i$ and reject those that violate the constraints. If it matches our inference-time reweighting, we will say so and credit the inequality constraint set rather than the integration. Two things could still separate the methods. The training-time loss may reshape the depth head, which post-hoc filtering cannot do. And filtering wastes candidates when most of the mass lies outside $F_i$, which is the inefficiency Bujnak et al. describe for classical solvers.
Why not just add more views? RayFusion shows that intersecting rays across agents works, but it needs communication, latency under 500 ms, and localization error under 0.6 m. Our setting is a single camera with no cooperation. Multi-view methods are out of scope and are not a competing answer to that setting.
Proposed Experiments
The main evaluation is KITTI 3D Car and Pedestrian $AP_{R40}$ on Easy, Moderate, and Hard, plus $AP_{BEV}$. We additionally stratify validation results by the dataset's occlusion and truncation labels, because the hypothesis predicts gains concentrated there. The decisive check is whether the interval-restricted MonoDETR beats both the reproduced baseline and the post-hoc rejection control on the occluded and truncated stratum. If it beats only the baseline, the contribution is the constraints. If it beats neither, the idea fails.
A second check tests calibration. Across objects, the width of $F_i$ should correlate with absolute depth error of the baseline detector, and the true depth should fall inside the soft interval at the rate implied by $\sigma$. If wide intervals do not mark hard objects, the geometry is not carrying information.
Third, we test robustness. We repeat Geo-Aug's protocol of perturbing object scale and vertical position, apply random crops that shift the principal point, and train on KITTI and test on nuScenes, comparing against Geo-Aug's reported 10.23 to 19.40 mAP transfer. Geometry computed from the true intrinsics should hold up under these shifts where learned cues do not. On the full nuScenes benchmark we report mAP and NDS, and we measure the added per-image latency, which should stay well under the 38 ms MonoDETR inference time.
Finally, inside MonoJSG we compare cost volumes over the fixed range and over $F_i$ at equal hypothesis counts. Restricting the range should either raise Moderate AP above its reported 16.14% or achieve it with fewer hypotheses.
References
- 3D Bounding Box Estimation Using Deep Learning and Geometry
- Lambda Twist: An Accurate Fast Robust Perspective Three Point (P3P) Solver
- Learning to Solve Hard Minimal Problems
- Exploring Geometric Consistency for Monocular 3D Object Detection
- MonoJSG: Joint Semantic and Geometric Cost Volume for Monocular 3D Object Detection
- Segmenting Epipolar Line
- Disentangling Monocular 3D Object Detection
- RayFusion: Ray Fusion Enhanced Collaborative Visual Perception
- MonoDETR: Depth-aware Transformer for Monocular 3D Object Detection
- MonoDETR: Depth-guided Transformer for Monocular 3D Object Detection
- Categorical Depth Distribution Network for Monocular 3D Object Detection
- Making minimal solvers fast