Object Segmentation by Long Term Analysis of Point Trajectories

Thomas Brox, Jitendra Malik

Abstract

Unsupervised learning requires a grouping step that defines which data belong together. A natural way of grouping in images is the segmentation of objects or parts of objects. While pure bottom-up segmentation from static cues is well known to be ambiguous at the object level, the story changes as soon as objects move. In this paper, we present a method that uses long term point trajectories based on dense optical flow. Defining pair-wise distances between these trajectories allows to cluster them, which results in temporally consistent segmentations of moving objects in a video shot. In contrast to multi-body factorization, points and even whole objects may appear or disappear during the shot. We provide a benchmark dataset and an evaluation method for this so far uncovered setting.
BibTeX
@inproceedings{brox2010object,
  author = {Thomas Brox and Jitendra Malik},
  title = {{Object Segmentation by Long Term Analysis of Point Trajectories}},
  booktitle = {European Conference on Computer Vision},
  year = {2010},
  doi = {10.1007/978-3-642-15555-0\_21},
}

Loading related content...

Figure