Towards Reliable Detection of Empty Space: Conditional Marked Point Processes for Object Detection

Tobias J. Riedlinger, Kira Maag, Hanno Gottschalk

TL;DR

Current object detectors are great at finding obstacles, but they cannot reliably quantify the probability that an undetected region is truly empty. This paper introduces a mathematically principled object detection framework based on Conditional Marked Point Processes (CMPPP) that provides well-calibrated confidence estimates for empty spaces, offering a crucial safety upgrade for autonomous driving and robotics.

What Problem Does This Paper Address?

In autonomous driving, assuming "no object detected means the path is safe" is a dangerous gamble. While modern deep learning models excel at predicting bounding boxes and assigning "objectness" scores to foreground objects, they fail to quantify uncertainty in the background. They do not provide a calibrated probability assessment of whether an empty area is truly free of obstacles.

While semantic segmentation models can classify pixels as "road," they treat pixel predictions as stochastically independent. When you multiply these independent pixel probabilities to assess a whole region, the math breaks down, leading to severe miscalibration and extreme underconfidence over larger areas. This paper bridges this safety gap by introducing a framework grounded in spatial statistics that natively understands regions and empty space.

Methods

The authors reframe object detection through the lens of spatial statistics. Instead of predicting arbitrary boxes, they treat bounding box data as realizations of a Marked Point Process.

Here is how it works:

  1. Points and Marks: The center of a bounding box is treated as a spatial point event. The properties of the box—its width, height, and class label—are treated as "marks" attached to that point.
  2. The CMPPP Model: They introduce a Conditional Marked Poisson Point Process (CMPPP). A neural network predicts a continuous spatial intensity function (where objects are likely to appear) alongside the mark distributions.
  3. Likelihood-Based Loss: Because standard Lebesgue measure density is ill-defined on infinite-dimensional point pattern spaces, the authors derive a loss function using Radon-Nikodym derivatives relative to a homogeneous Poisson point process. Remarkably, this mathematically rigorous derivation naturally yields familiar components: a distributed objectness score, an $L_1$ loss for box regression (under a Laplace distribution assumption), and cross-entropy for classification.
  4. Inference Without Standard NMS: Instead of traditional Non-Maximum Suppression (NMS), the model uses the counting statistics of the point process to estimate the expected number of objects and extracts peaks from the intensity landscape.
Figure 1: Left: Semantic segmentation prediction. Center left: Poisson point process intensity. Center right: Conditional marked Poisson point process intensity. Right: Bounding box prediction.

Datasets

The framework was evaluated on three distinct datasets representing practical use cases for empty space detection:

  • Cityscapes: Dense urban traffic scenes used to evaluate drivable space for autonomous cars.
  • TiROD (Tiny Robotics Object Detection): Indoor lab/office and outdoor garden scenes captured from a frog's perspective for robotics navigation.
  • VisDrone: Aerial drone imagery where identifying free landing sites is critical.

Key Insights and Ideas

  • The Independence Trap: Standard models suffer from exponential scaling errors when calculating "void confidence" (the probability that a region is empty) because they multiply independent pixel probabilities. CMPPP avoids this by using a cumulative logic that integrates intensity over an entire area.
  • Zero Training Hyperparameters: By using maximum likelihood equations to determine the scale parameter ($\sigma$) of the box regression, the model trains end-to-end with zero tuning hyperparameters and only a single inference hyperparameter (the crop patch size for peak extraction).
  • True Region-Based Confidence: Because the model is built on counting processes, it can analytically compute the exact probability that an arbitrary test region (of any shape or size) contains zero object centers or intersections.

Key Results

The primary strength of the CMPPP framework is its exceptional calibration performance, measured via Expected Calibration Error (ECE).

  • Empty Space Calibration: On the Cityscapes dataset, the CMPPP method achieved ECE values close to the ideal diagonal, outperforming standard semantic segmentation baselines by up any order of magnitude. For a box size of 1,000, DeepLabv3+ had a semantic ECE of 0.2245, while the proposed PPP method dropped the error to a striking 0.0029.
Confidence calibration curves comparing the proposed PPP method against standard semantic segmentation.
  • Detection Performance: While the model's primary goal is reliable calibration rather than beating state-of-the-art detection accuracy, it remains highly competitive. On Cityscapes, the HRNet-backed CMPPP achieved a mean Average Precision (mAP) of 55.49%, closely trailing standard uncalibrated detectors like Faster R-CNN (59.32%).
  • Efficiency: The model maintains the runtime complexity of standard semantic segmentation networks, making it highly scalable for real-time robotics applications.

Limitations and Future Work

The authors note a few areas for improvement:

  • Scale Miscalibration: The model currently struggles with very large foreground objects because it assigns a fixed square patch size during peak inference, which can split the spread-out intensity of a large object. Incorporating depth-dependent patch sizes is a suggested fix.
  • Lack of Repelling Potentials: The underlying Poisson assumption implies that object configurations are non-interacting. In reality, physical objects cannot overlap. Introducing "repelling potentials" (e.g., Gibbs point processes) could make the spatial statistics even more realistic, though at the cost of higher computational complexity.

Summary updated

Abstract

Deep neural networks have set the state-of-the-art in computer vision tasks such as bounding box detection and semantic segmentation. Object detectors and segmentation models assign confidence scores to predictions, reflecting the model's uncertainty in object detection or pixel-wise classification. However, these confidence estimates are often miscalibrated, as their architectures and loss functions are tailored to task performance rather than probabilistic foundation. Even with well calibrated predictions, object detectors fail to quantify uncertainty outside detected bounding boxes, i.e., the model does not make a probability assessment of whether an area without detected objects is truly free of obstacles. This poses a safety risk in applications such as automated driving, where uncertainty in empty areas remains unexplored. In this work, we propose an object detection model grounded in spatial statistics. Bounding box data matches realizations of a marked point process, commonly used to describe the probabilistic occurrence of spatial point events identified as bounding box centers, where marks are used to describe the spatial extension of bounding boxes and classes. Our statistical framework enables a likelihood-based training and provides well-defined confidence estimates for whether a region is drivable, i.e., free of objects. We demonstrate the effectiveness of our method through calibration assessments and evaluation of performance.
BibTeX
@inproceedings{riedlinger2026towards,
  author = {Tobias J. Riedlinger and Kira Maag and Hanno Gottschalk},
  title = {{Towards Reliable Detection of Empty Space: Conditional Marked Point Processes for Object Detection}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=M2KLWLHzX0},
}
Figure