Semi-supervisedly Co-embedding Attributed Networks

Zaiqiao Meng, Shangsong Liang, Jinyuan Fang, Teng Xiao

Abstract

Deep generative models (DGMs) have achieved remarkable advances. Semi-supervised variational auto-encoders (SVAE) as a classical DGM offer a principled framework to effectively generalize from small labelled data to large unlabelled ones, but it is difficult to incorporate rich unstructured relationships within the multiple heterogeneous entities. In this paper, to deal with the problem, we present a semi-supervised co-embedding model for attributed networks (SCAN) based on the generalized SVAE for heterogeneous data, which collaboratively learns low-dimensional vector representations of both nodes and attributes for partially labelled attributed networks semi-supervisedly. The node and attribute embeddings obtained in a unified manner by our SCAN can benefit for capturing not only the proximities between nodes but also the affinities between nodes and attributes. Moreover, our model also trains a discriminative network to learn the label predictive distribution of nodes. Experimental results on real-world networks demonstrate that our model yields excellent performance in a number of applications such as attribute inference, user profiling and node classification compared to the state-of-the-art baselines.

Introduction

Attributed networks, where nodes represent entities and edges represent relationships, are ubiquitous in real-world applications. These networks often contain rich node features or attributes. However, existing embedding methods frequently encounter three significant challenges. First, they often exhibit a node-centric bias, treating attributes merely as auxiliary inputs rather than entities to be embedded, which hinders the direct measurement of node-attribute affinities. Second, many methods are purely unsupervised and fail to leverage available partial labels to guide the learning process. Finally, most models represent nodes as single points in a latent space, failing to account for the uncertainty inherent in the data.

To address these limitations, we propose SCAN (Semi-supervised Co-embedding Attributed Network). SCAN extends Semi-supervised Variational Auto-encoders (SVAE) to heterogeneous data, co-embedding both nodes and attributes as Gaussian distributions in a shared semantic space.

Methodology

The SCAN framework is built upon a generalized SVAE architecture designed for heterogeneous data. It consists of an inference network (encoder) and a generative network (decoder).

The SCAN architecture illustrating the flow from input data to reconstructed data through inference and generative networks.

Co-embedding and Gaussian Representations

Unlike traditional models, SCAN maps nodes and attributes into the same low-dimensional space. Each entity is represented by a Gaussian distribution, where the mean denotes its semantic location and the variance captures the uncertainty. This shared space allows for direct similarity measurements between nodes and attributes.

Five-Case Variational Evidence Lower Bound (ELBO)

To handle the semi-supervised nature of the data, we derive an ELBO that accounts for five distinct observation types, ranging from edges between labeled nodes to attribute associations for unlabeled nodes. The model utilizes the Gumbel-Softmax trick to handle discrete labels, enabling differentiable backpropagation through the categorical label distribution.

Experimental Evaluation

We evaluated SCAN on three benchmark datasets: Pubmed (citation network), BlogCatalog (social network with keywords), and Flickr (social network with interest groups).

Node Classification

SCAN consistently outperformed state-of-the-art baselines including GCN, Planetoid, and SEANO. As shown in the results, SCAN achieves superior accuracy and F1 scores across all datasets, particularly on BlogCatalog where it reached an accuracy of 0.839.

Performance comparison of SCAN against state-of-the-art baselines for node classification.

Attribute Inference and Uncertainty

In attribute inference tasks, SCAN demonstrated a significant lead. For instance, on the Pubmed dataset, it achieved an AUC of 0.713, outperforming the CAN baseline (0.670). Furthermore, the Gaussian representation allows for the visualization of uncertainty; nodes with fewer connections or sparse features typically exhibit larger variances (ellipsoids), providing better interpretability.

Conclusion

SCAN provides a robust framework for semi-supervised learning on attributed networks by co-embedding nodes and attributes. While the model excels in static environments, future work will explore its extension to dynamic networks and more complex heterogeneous graphs with multiple node and edge types.

Summary updated

Abstract

Deep generative models (DGMs) have achieved remarkable advances. Semi-supervised variational auto-encoders (SVAE) as a classical DGM offer a principled framework to effectively generalize from small labelled data to large unlabelled ones, but it is difficult to incorporate rich unstructured relationships within the multiple heterogeneous entities. In this paper, to deal with the problem, we present a semi-supervised co-embedding model for attributed networks (SCAN) based on the generalized SVAE for heterogeneous data, which collaboratively learns low-dimensional vector representations of both nodes and attributes for partially labelled attributed networks semi-supervisedly. The node and attribute embeddings obtained in a unified manner by our SCAN can benefit for capturing not only the proximities between nodes but also the affinities between nodes and attributes. Moreover, our model also trains a discriminative network to learn the label predictive distribution of nodes. Experimental results on real-world networks demonstrate that our model yields excellent performance in a number of applications such as attribute inference, user profiling and node classification compared to the state-of-the-art baselines.
BibTeX
@inproceedings{meng2019semi,
  author = {Zaiqiao Meng and Shangsong Liang and Jinyuan Fang and Teng Xiao},
  title = {{Semi-supervisedly Co-embedding Attributed Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=8PAC_Wa-nr},
}
Figure