Semi-Supervised Temporal Action Proposal Generation via Exploiting 2-D Proposal Map

Weining Wang, Tianwei Lin, Dongliang He, Fu Li, Shilei Wen, Liang Wang, Jing Liu

Abstract

Temporal action proposal generation aims to generate temporal video segments containing human actions in untrimmed videos, which is always a preliminary for such video understanding tasks as action localization and temporally description grounding, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> . Fully-supervised solutions, though proven to be effective, suffer much from heavy data annotation overhead. To address this problem, this paper focuses on a rarely investigated yet practical problem of semi-supervised learning for temporal action proposal generation. Firstly, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Proposal Map oriented Mean-Teacher</i> (PM-MT) model, which can use both labeled and unlabeled data for end-to-end model training. Secondly, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Suppression-and-Re-Generation</i> (SRG) strategy is designed to generate high-quality pseudo labels for unlabeled data, which are then used to finetune the model. Extensive experiments demonstrate the effectiveness of our proposed method, by achieving the state-of-the-art results on two public benchmark datatsets on the task of semi-supervised action proposal generation and outperforming fully-supervised learning methods with only a portion of labeled data.
BibTeX
@article{wang2022semi,
  author = {Weining Wang and Tianwei Lin and Dongliang He and Fu Li and Shilei Wen and Liang Wang and Jing Liu},
  title = {{Semi-Supervised Temporal Action Proposal Generation via Exploiting 2-D Proposal Map}},
  journal = {IEEE Trans. Multimedia},
  year = {2022},
  url = {https://openreview.net/forum?id=8guc2PdUbpy},
}

Loading related content...

Figure