Important Scene Detection Of Baseball Videos Via Time-Lag Aware Deep Multiset Canonical Correlation Maximization

Kaito Hirasawa, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

Abstract

This paper presents a new important scene detection method of baseball videos based on correlation maximization between heterogeneous modalities via time-lag aware deep multiset canonical correlation analysis (Tl-dMCCA). The technical contributions of this paper are twofold. First, textual, visual and audio features calculated from tweets and videos are adopted as multi-view time series features. Since Tl-dMCCA which utilizes these features includes the unsupervised embedding scheme via deep networks, the proposed method can flexibly express the relationship between heterogeneous features. Second, since there is the time-lag between posted tweets and the corresponding multiple previous events, Tl-dMCCA considers the time-lag relationships between them. Specifically, we newly introduce the representation of such time-lags into the derivation of their covariance matrices. By considering time-lags via Tl-dMCCA, the proposed method correctly detects important scenes.
BibTeX
@inproceedings{hirasawa2020important,
  author = {Kaito Hirasawa and Keisuke Maeda and Takahiro Ogawa and Miki Haseyama},
  title = {{Important Scene Detection Of Baseball Videos Via Time-Lag Aware Deep Multiset Canonical Correlation Maximization}},
  booktitle = {ICIP},
  year = {2020},
  doi = {10.1109/icip40778.2020.9191070},
}

Loading related content...

Figure