Improving cross-modal correlation learning with hyperlinks

Shuhui Wang, Yiling Wu, Qingming Huang

Abstract

We propose a new cross-modal correlation learning framework which boosts the performance of correlation learning models using the hyperlink information. First, we design a neighborhood selection paradigm using the hyperlink structure and content similarities to identify a set of semantically related documents for each multi-modal document in both training and testing stage. Based on the neighborhood structure, we revise two well-established content-based correlation learning models, i.e., canonical correlation analysis (CCA) and kernel canonical correlation analysis (KCCA) with a structure coding matrix. Third, we develop a correlation score aggregation technique to discover more semantically relevant cross-modal documents. To our best knowledge, this is the first to introduce hyperlink information into cross-modal correlation learning. Experimental results demonstrate that our proposed framework can significantly improve the model generality towards real-world cross-modal retrieval.
BibTeX
@inproceedings{wang2015improving,
  author = {Shuhui Wang and Yiling Wu and Qingming Huang},
  title = {{Improving cross-modal correlation learning with hyperlinks}},
  booktitle = {ICME},
  year = {2015},
  doi = {10.1109/icme.2015.7177411},
}

Loading related content...

Figure