Object proposal generation applying the distance dependent Chinese restaurant process

Mikko Lauri, Simone FrintroparXiv 2017 · arXiv

TL;DR

This paper introduces a non-parametric Bayesian approach to object proposal generation using the distance dependent Chinese restaurant process (ddCRP). By sampling multiple possible image segmentations rather than relying on a single "best guess," the method provides a robust likelihood estimate for each proposal, significantly improving the ranking and quality of discovered objects in cluttered robotic environments.

What Problem Does This Paper Address?

In robotics, uncertainty is a constant. A robot navigating a kitchen needs to identify objects for grasping, but lighting changes and motion blur make deterministic "one-shot" vision algorithms unreliable. Most current object discovery methods produce a single set of segments without any measure of how "sure" the algorithm is about them.

The authors argue that for a robot to make informed decisions—like choosing which object is safest to grab—it needs the full Bayesian posterior probability distribution. In other words, the robot shouldn't just ask "What is an object?" but "How likely is it that this specific segment corresponds to a real-world object?"

Methods

The authors propose a multi-step pipeline that moves from raw pixels to ranked object candidates:

  1. Oversegmentation: The image is first broken down into "superpixels" (small, homogeneous clusters of pixels) using the SLIC algorithm.
  2. The ddCRP Prior: They apply the distance dependent Chinese restaurant process. Imagine superpixels as "customers" in a restaurant. Each customer chooses to sit at a "table" (an object cluster) based on their distance to others. In this model, "distance" isn't just spatial; it includes similarities in color and intensity.
  3. The ddCRP process: links between superpixels (customers) induce table assignments (object segments).
  4. MCMC Sampling: Because calculating the perfect segmentation is mathematically "intractable" (too complex), they use Markov chain Monte Carlo (MCMC) sampling. They draw multiple different segmentation "samples" from the posterior distribution.
  5. Likelihood Estimation: This is the core innovation. By looking across all generated samples, they calculate a likelihood term $P(o)$ for each proposal. If a group of superpixels appears together in many different samples, it’s highly likely to be a real object.
  6. Ranking with Gestalt Principles: Finally, they rank these proposals using a Support Vector Machine (SVM) trained on human-centric "Gestalt" features like symmetry, convexity, and compactness, weighted by the new Bayesian likelihood term.
Fig. 2. The distance dependent Chinese restaurant process. Customers corresponding to superpixels in the input image are denoted by the nodes xi. The links between customers induce a table assignment which corresponds to a segmentation of the image.
Fig. 1. Overview of the object discovery approach. Superpixels in an initial overseg- mentation (1) are grouped applying the distance dependent Chinese restaurant process (ddCRP) (2). Multiple segmentation samples are drawn from the ddCRP posterior dis- tribution. Object proposals are extracted from the set of segmentation samples (3), and ranked according to how likely they correspond to an object (4).

Datasets

The method was evaluated on the Kitchen Object Discovery (KOD) dataset.

  • Why KOD? Unlike standard datasets that feature single, clear objects, KOD contains sequences of cluttered indoor scenes with roughly 80 objects per sequence. This mimics the actual challenges a mobile robot faces in a home environment.

Key Insights and Ideas

  • Likelihood as a Filter: The "aha moment" is that the frequency with which a segment appears in random samples is a powerful indicator of its quality. A segment that only appears once is likely noise; a segment that appears in 90% of samples is likely a physical object.
  • Encoding Human Perception: By setting ddCRP hyperparameters to favor internal consistency (similar colors) and external contrast (different from neighbors), the authors bake "Gestalt principles" directly into the Bayesian prior.
  • Diversity of Proposals: Unlike deterministic methods that might miss an object entirely if their one guess is wrong, the sampling approach generates a diverse set of candidates, increasing the chance that the true object is "caught" in at least one sample.

Key Results

The researchers compared their "weighted" (likelihood-aware) method against several state-of-the-art baselines, including Saliency-Guided Object candidates (SGO) and Randomized Prim's (RP).

  • Superior Precision and Recall: The proposed method achieved the highest average precision (20.8) and recall (78.3) across the test sequences.
  • The Power of Weighting: Using the likelihood term to rank proposals significantly outperformed "plain" scoring. As seen in the Kitchen C sequence, the Bayesian approach achieves much higher precision for the top 50 proposals compared to other methods.
  • Qualitative Success: In cluttered scenes (like a table full of mugs and plates), the likelihood weighting successfully prioritized actual objects (like a coffee cup) over background patches (like a piece of the floor) that might otherwise look "object-like" to simpler algorithms.
Comparison of ranking methods: Likelihood weighting (red/dashed) consistently improves recall and global detection.

Limitations and Future Work

While the results are promising, the authors note a few areas for growth:

  • Parameter Dependency: The performance relies on tuning hyperparameters (like the decay function $f$). A more automated way to set these would be beneficial.
  • Data Modality: The current work focuses on standard 2D images. The authors suggest that extending the method to RGB-D (depth) data would likely yield even better results, as physical distance in 3D space is a much stronger cue for objectness than 2D pixel distance.
  • Computational Cost: While not detailed extensively, MCMC sampling is generally more computationally expensive than deterministic methods, which might be a concern for real-time robotics.

Summary updated

Abstract

In application domains such as robotics, it is useful to represent the uncertainty related to the robot's belief about the state of its environment. Algorithms that only yield a single "best guess" as a result are not sufficient. In this paper, we propose object proposal generation based on non-parametric Bayesian inference that allows quantification of the likelihood of the proposals. We apply Markov chain Monte Carlo to draw samples of image segmentations via the distance dependent Chinese restaurant process. Our method achieves state-of-the-art performance on an indoor object discovery data set, while additionally providing a likelihood term for each proposal. We show that the likelihood term can effectively be used to rank proposals according to their quality.
BibTeX
@misc{lauri2017object,
  author = {Mikko Lauri and Simone Frintrop},
  title = {{Object proposal generation applying the distance dependent Chinese restaurant process}},
  year = {2017},
  eprint = {1704.03706},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1704.03706},
}
Figure