Collaborative Knowledge Distillation for Heterogeneous Information Network Embedding

Can Wang, Sheng Zhou, Kang Yu, Defang Chen, Bolang Li, Yan Feng, Chun Chen

Abstract

Learning low-dimensional representations for Heterogeneous Information Networks (HINs) has drawn increasing attention recently for its effectiveness in real-world applications. Compared with homogeneous networks, HINs are characterized by meta-paths connecting different types of nodes with semantic meanings. Existing methods mainly follow the prototype of independently learning meta-path-based embeddings and integrating them into a unified embedding. However, meta-paths in a HIN are inherently correlated since they reflect different perspectives of the same object. If each meta-path is treated as an isolated semantic data resource and the correlations among them are disregarded, sub-optimality in the both the meta-path based embedding and final embedding will be resulted. To address this issue, we make the first attempt to explicitly model the correlation among meta-paths by proposing Collaborative Knowledge Distillation for Heterogeneous Information Network Embedding (CKD). More specifically, we model the knowledge in each meta-path with two different granularities: regional knowledge and global knowledge. We learn the meta-path-based embeddings by collaboratively distill the knowledge from intra-meta-path and inter-meta-path simultaneously. Experiments conducted on six real-world HIN datasets demonstrates the effectiveness of the CKD method.
BibTeX
@inproceedings{wang2022collaborative,
  author = {Can Wang and Sheng Zhou and Kang Yu and Defang Chen and Bolang Li and Yan Feng and Chun Chen},
  title = {{Collaborative Knowledge Distillation for Heterogeneous Information Network Embedding}},
  booktitle = {The Web Conference},
  year = {2022},
  doi = {10.1145/3485447.3512209},
}

Loading related content...

Figure