Sparse Multi-Modal Hashing

Fei Wu, Zhou Yu, Yi Yang, Siliang Tang, Yin Zhang, Yueting Zhuang

Abstract

Learning hash functions across heterogenous high-dimensional features is very desirable for many applications involving multi-modal data objects. In this paper, we propose an approach to obtain the sparse codesets for the data objects across different modalities via joint multi-modal dictionary learning, which we call sparse multi-modal hashing (abbreviated as <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">${\rm SM}^{2}{\rm H}$</tex></formula> ). In <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">${\rm SM}^{2}{\rm H}$</tex></formula> , both intra-modality similarity and inter-modality similarity are first modeled by a hypergraph, then multi-modal dictionaries are jointly learned by Hypergraph Laplacian sparse coding. Based on the learned dictionaries, the sparse codeset of each data object is acquired and conducted for multi-modal approximate nearest neighbor retrieval using a sensitive Jaccard metric. The experimental results show that <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">${\rm SM}^{2}{\rm H}$</tex></formula> outperforms other methods in terms of mAP and Percentage on two real-world data sets.
BibTeX
@article{wu2014sparse,
  author = {Fei Wu and Zhou Yu and Yi Yang and Siliang Tang and Yin Zhang and Yueting Zhuang},
  title = {{Sparse Multi-Modal Hashing}},
  journal = {IEEE Trans. Multimedia},
  year = {2014},
  doi = {10.1109/tmm.2013.2291214},
}

Loading related content...

Figure