Object co-segmentation via discriminative low rank matrix recovery

Yong Li, Jing Liu, Zechao Li, Yang Liu, Hanqing LuACM Multimedia 2013 · DOI

TL;DR

This paper introduces a novel approach to object co-segmentation that tackles the challenge of distinguishing true objects from salient but irrelevant background regions. By combining low-rank matrix recovery for saliency detection with discriminative learning for commonality, their method achieves state-of-the-art performance on benchmark datasets.

What Problem Does This Paper Address?

Object co-segmentation aims to identify and segment the same object class across a set of images. Traditional methods often assume that regions common to multiple images are the objects of interest. However, this assumption breaks down when backgrounds are also consistent across images, leading to misidentification. For instance, in a set of baseball images, the grass in the outfield might be common but is not the object of interest (the baseball player). This paper addresses the critical need to differentiate between common object regions and common background regions, even when those backgrounds are salient.

Methods

The core of the proposed method is the Discriminative Low Rank Matrix Recovery (DLRR) algorithm. It operates in three main stages:

  1. Saliency Detection via Low-Rank Matrix Recovery: Inspired by prior work, the authors represent each image's super-pixels as a feature matrix. They assume that background regions can be represented by a low-rank matrix, while salient regions manifest as sparse "noise." By decomposing the feature matrix into a low-rank component (background) and a sparse component (salient regions), they can identify potential salient areas. They further incorporate a "high-level prior" (like distance to the image center) to refine saliency detection and reduce spurious small salient regions.
  2. Discriminative Learning for Commonality: To ensure that identified regions are not just salient but also common across images, a discriminative learning component is employed. This part uses a logistic regression model to predict the probability of each super-pixel belonging to the target object class, leveraging the feature representations.
  3. Joint Optimization and Regularization: The key innovation lies in jointly optimizing these two components. A regularization penalty is introduced to measure the disagreement between the predicted saliency (from low-rank recovery) and the object probability (from discriminative learning). This encourages the two parts to reinforce each other. The overall objective function is then solved using an efficient block-coordinate descent optimization procedure.

Datasets

The authors evaluate their method on two publicly available datasets:

  • MSRC-v2: A dataset containing images of various object classes.
  • iCoseg: Another dataset for object co-segmentation.

Key Insights and Ideas

  • Challenging the "Common = Object" Assumption: The paper directly confronts the limitation of previous co-segmentation methods by highlighting and addressing the issue of consistent, salient backgrounds.
  • Synergy of Saliency and Commonality: The core idea is that true objects are both salient within an image and common across a set of images. By jointly learning these two properties, the method becomes more robust.
  • Low-Rank for Background, Sparse for Salient: The elegant mathematical formulation of representing background as low-rank and salient regions as sparse provides a principled way to extract saliency information.
  • Regularization for Joint Learning: The introduction of a specific regularization term to enforce agreement between saliency and discriminative cues is crucial for the method's success.

Key Results

The DLRR method demonstrates superior performance compared to several state-of-the-art approaches on both the MSRC and iCoseg datasets.

  • MSRC Dataset: DLRR achieved the best average performance and outperformed other methods on 8 out of 14 classes. It also showed a lower standard deviation, indicating greater robustness.
  • iCoseg Dataset: DLRR achieved the best performance on 5 out of 10 image classes and showed a significant average segmentation score increase (12.7%) compared to one of the baseline methods.
  • Qualitative Results: The paper includes visual examples showcasing successful segmentations and also highlights some failed cases, attributing them to extreme diversity in object appearance.

Limitations and Future Work

  • Limitations: The paper notes that some failed results may be due to the extreme diversity of object appearance within a class. The method's performance might be sensitive to the quality of the initial super-pixel segmentation.
  • Future Work: Not explicitly detailed, but the success on these datasets suggests potential for application in more complex scenarios or with larger, more diverse datasets. The authors' focus on improving robustness to background clutter could be extended to other vision tasks.

Summary updated

BibTeX
@inproceedings{li2013object,
  author = {Yong Li and Jing Liu and Zechao Li and Yang Liu and Hanqing Lu},
  title = {{Object co-segmentation via discriminative low rank matrix recovery}},
  booktitle = {ACM Multimedia},
  year = {2013},
  doi = {10.1145/2502081.2502195},
}
Figure