Abstract
Deep generative models (DGMs) have achieved remarkable advances. Semi-supervised variational auto-encoders (SVAE) as a classical DGM offer a principled framework to effectively generalize from small labelled data to large unlabelled ones, but it is difficult to incorporate rich unstructured relationships within the multiple heterogeneous entities. In this paper, to deal with the problem, we present a semi-supervised co-embedding model for attributed networks (SCAN) based on the generalized SVAE for heterogeneous data, which collaboratively learns low-dimensional vector representations of both nodes and attributes for partially labelled attributed networks semi-supervisedly. The node and attribute embeddings obtained in a unified manner by our SCAN can benefit for capturing not only the proximities between nodes but also the affinities between nodes and attributes. Moreover, our model also trains a discriminative network to learn the label predictive distribution of nodes. Experimental results on real-world networks demonstrate that our model yields excellent performance in a number of applications such as attribute inference, user profiling and node classification compared to the state-of-the-art baselines.
Introduction
Attributed networks, where nodes represent entities and edges represent relationships, are ubiquitous in real-world applications. These networks often contain rich node features or attributes. However, existing embedding methods frequently encounter three significant challenges. First, they often exhibit a node-centric bias, treating attributes merely as auxiliary inputs rather than entities to be embedded, which hinders the direct measurement of node-attribute affinities. Second, many methods are purely unsupervised and fail to leverage available partial labels to guide the learning process. Finally, most models represent nodes as single points in a latent space, failing to account for the uncertainty inherent in the data.
To address these limitations, we propose SCAN (Semi-supervised Co-embedding Attributed Network). SCAN extends Semi-supervised Variational Auto-encoders (SVAE) to heterogeneous data, co-embedding both nodes and attributes as Gaussian distributions in a shared semantic space.
Methodology
The SCAN framework is built upon a generalized SVAE architecture designed for heterogeneous data. It consists of an inference network (encoder) and a generative network (decoder).
Co-embedding and Gaussian Representations
Unlike traditional models, SCAN maps nodes and attributes into the same low-dimensional space. Each entity is represented by a Gaussian distribution, where the mean denotes its semantic location and the variance captures the uncertainty. This shared space allows for direct similarity measurements between nodes and attributes.
Five-Case Variational Evidence Lower Bound (ELBO)
To handle the semi-supervised nature of the data, we derive an ELBO that accounts for five distinct observation types, ranging from edges between labeled nodes to attribute associations for unlabeled nodes. The model utilizes the Gumbel-Softmax trick to handle discrete labels, enabling differentiable backpropagation through the categorical label distribution.
Experimental Evaluation
We evaluated SCAN on three benchmark datasets: Pubmed (citation network), BlogCatalog (social network with keywords), and Flickr (social network with interest groups).
Node Classification
SCAN consistently outperformed state-of-the-art baselines including GCN, Planetoid, and SEANO. As shown in the results, SCAN achieves superior accuracy and F1 scores across all datasets, particularly on BlogCatalog where it reached an accuracy of 0.839.
Attribute Inference and Uncertainty
In attribute inference tasks, SCAN demonstrated a significant lead. For instance, on the Pubmed dataset, it achieved an AUC of 0.713, outperforming the CAN baseline (0.670). Furthermore, the Gaussian representation allows for the visualization of uncertainty; nodes with fewer connections or sparse features typically exhibit larger variances (ellipsoids), providing better interpretability.
Conclusion
SCAN provides a robust framework for semi-supervised learning on attributed networks by co-embedding nodes and attributes. While the model excels in static environments, future work will explore its extension to dynamic networks and more complex heterogeneous graphs with multiple node and edge types.
Summary updated