Multi-layer multi-instance kernel for video concept detection

Zhiwei Gu, Tao Mei, Xian-Sheng Hua, Jinhui Tang, Xiuqing Wu

Abstract

In video concept detection, most existing methods have not well studied the intrinsic hierarchical structure of video content. However, unlike flat attribute-value data used in many existing methods, video is essentially a structured media with multi-layer representation. For example, a video can be represented by a hierarchical structure including, from large to small, shot, key-frame, and region. Moreover, it fits the typical Multi-Instance (MI) setting in which the "bag-instance" correspondence is embedded among contiguous layers. We call such multi-layer structure and the "bag-instance" relation embedded in the structure as Multi-Layer Multi-Instance (MLMI) setting in this paper. We formulate video concept detection as an MLMI learning problem in which a rooted tree with MLMI nature embedded is devised to represent a video segment. Furthermore, by fusing the information from different layers, we construct a novel MLMI kernel to measure the similarities between the instances in the same and different layers. In contrast to traditional MI learning, both the Multi-Layer structure and Multi-Instance relations are leveraged simultaneously in the proposed kernel. We applied MLMI kernel to concept detection task on TRECVID 2005 corpus and reported superior performance (+25% in Mean Average Precision) to standard Support Vector Machine based approaches.
BibTeX
@inproceedings{gu2007multi,
  author = {Zhiwei Gu and Tao Mei and Xian-Sheng Hua and Jinhui Tang and Xiuqing Wu},
  title = {{Multi-layer multi-instance kernel for video concept detection}},
  booktitle = {ACM Multimedia},
  year = {2007},
  doi = {10.1145/1291233.1291309},
}

Loading related content...

Figure