Track to the future: Spatio-temporal video segmentation with long-range motion cues

J. Lezama, Karteek Alahari, Josef Sivic, Ivan Laptev

Abstract

Video provides not only rich visual cues such as motion and appearance, but also much less explored long-range temporal interactions among objects. We aim to capture such interactions and to construct a powerful intermediate-level video representation for subsequent recognition. Motivated by this goal, we seek to obtain spatio-temporal oversegmentation of a video into regions that respect object boundaries and, at the same time, associate object pixels over many video frames. The contributions of this paper are two-fold. First, we develop an efficient spatiotemporal video segmentation algorithm, which naturally incorporates long-range motion cues from the past and future frames in the form of clusters of point tracks with coherent motion. Second, we devise a new track clustering cost function that includes occlusion reasoning, in the form of depth ordering constraints, as well as motion similarity along the tracks. We evaluate the proposed approach on a challenging set of video sequences of office scenes from feature length movies.
BibTeX
@inproceedings{lezama2011track,
  author = {J. Lezama and Karteek Alahari and Josef Sivic and Ivan Laptev},
  title = {{Track to the future: Spatio-temporal video segmentation with long-range motion cues}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2011},
  doi = {10.1109/cvpr.2011.6044588},
}

Loading related content...

Figure