Leveraging Conceptualization for Short-Text Embedding

Heyan Huang, Yashen Wang, Chong Feng, Zhirun Liu, Qiang Zhou

Abstract

Most short-text embedding models typically represent each short-text only using the literal meanings of the words, which makes these models indiscriminative for the ubiquitous polysemy. In order to enhance the semantic representation capability of the short-texts, we (i) propose a novel short-text conceptualization algorithm to assign the associated concepts for each short-text, and then (ii) introduce the conceptualization results into learning the conceptual short-text embeddings. Hence, this semantic representation is more expressive than some widely-used text representation models such as the latent topic model. Wherein, the short-text conceptualization algorithm used here is based on a novel co-ranking framework, enabling the signals (i.e., the words and the concepts) to fully interplay to derive the solid conceptualization for the short-texts. Afterwards, we further extend the conceptual short-text embedding models by utilizing an attention-based model that selects the relevant words within the context to make more efficient prediction. The experiments on the real-world datasets demonstrate that the proposed conceptual short-text embedding model and short-text conceptualization algorithm are more effective than the state-of-the-art methods.
BibTeX
@article{huang2018leveraging,
  author = {Heyan Huang and Yashen Wang and Chong Feng and Zhirun Liu and Qiang Zhou},
  title = {{Leveraging Conceptualization for Short-Text Embedding}},
  journal = {IEEE Trans. Knowl. Data Eng.},
  year = {2018},
  url = {https://openreview.net/forum?id=TAXATNnoSV},
}

Loading related content...

Figure