Deep Cross-Modal Face Naming for People News Retrieval

Yong Tian, Lian Zhou, Yuejie Zhang, Tao Zhang, Weiguo Fan

Abstract

How to integrate multimodal information sources for face naming in multimodal news is a hot and yet challenging problem. A novel deep cross-modal face naming scheme is developed in this paper to facilitate more effective people news retrieval for large-scale multimodal news. This scheme integrates deep multimodal analysis, cross-modal correlation learning, and multimodal information mining, in which the efficient naming mechanism aims to cluster the deep features of different modalities into a common space to explore their inter-related correlations, and a special Web mining pattern is designed to optimize the name-face matching for rare non-celebrity. Such a cross-modal face naming model can be treated as a problem of bi-media semantic mapping and modeled as an inter-related correlation distribution over deep representations of multimodal news, in which the most important is to create more effective cross-modal name-face correlation and measure to what degree they are correlated. The experiments on a large number of public data from Yahoo! News have obtained very positive results and demonstrated the effectiveness of the proposed model.
BibTeX
@article{tian2021deep,
  author = {Yong Tian and Lian Zhou and Yuejie Zhang and Tao Zhang and Weiguo Fan},
  title = {{Deep Cross-Modal Face Naming for People News Retrieval}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2021},
  doi = {10.1109/tkde.2019.2948875},
}

Loading related content...

Figure