Multi-modal tag localization for mobile video search
Lists
Only you see this
Abstract
Given the tremendous growth of mobile videos, video tag localization, which localizes the relevant video clips for an associated semantic tag, is becoming increasingly important to influence users browsing and searching experience. However, most existing approaches adopt and depend to large degree on carefully selected visual features, which are manually designed by experts and do not take multi-modality into consideration. Aiming to take into account complementarity of different modalities, in this paper, we propose a multi-modal tag localization framework by exploiting deep learning to learn visual, auditory, and semantic features of videos for tag localization. Furthermore, we showcase that the framework can be applied to two novel mobile video search applications: (1) automatic time-code-level tags generation and (2) query-dependent video thumbnail selection. Extensive experiments on the public dataset show that the proposed approach achieves promising results, which obtains \(7.6~\%\) improvement beyond the state-of-the-arts. Finally, the subjective evaluation of usability demonstrates the proposed applications can significantly improve the user’s mobile video search experience.
BibTeX
@inproceedings{zhang2017multi,
author = {Rui Zhang and Sheng Tang and Wu Liu and Yongdong Zhang and Jintao Li},
title = {{Multi-modal tag localization for mobile video search}},
booktitle = {Multim. Syst},
year = {2017},
url = {https://openreview.net/forum?id=svIXbtNE5l},
}Loading related content...