Adversarial Mutual Information Learning for Network Embedding

Dongxiao He, Lu Zhai, Zhigang Li, Di Jin, Liang Yang, Yuxiao Huang, Philip S. Yu

Abstract

Adversarial learning in network embedding often focuses on forcing latent representations to follow a prior distribution. This paper introduces Adversarial Mutual Information Learning (AMIL), which shifts the focus to the encoding mechanism itself. By leveraging mutual information as a proxy for mapping quality and introducing a competitive negative sample generator, AMIL produces robust node representations that significantly outperform state-of-the-art methods in classification and clustering tasks.

1. Introduction

Network embedding is a fundamental task in graph analysis, aiming to map nodes into low-dimensional vectors while preserving structural and attribute information. While Generative Adversarial Networks (GANs) have been integrated into this field, most existing approaches use the discriminator primarily to regularize the latent space toward a specific prior distribution, such as a Gaussian. This paper argues that such methods fail to exploit the full potential of adversarial learning. Instead, the authors propose Adversarial Mutual Information Learning (AMIL), which focuses on optimizing the encoding mechanism itself.

2. Methodology

The AMIL framework is built upon a minimax game involving three primary components:

  1. The Autoencoder: Utilizing a Graph Convolutional Network (GCN) as an encoder, it maps node attributes ($X$) and topology ($A$) into an embedding ($Z$). An inner-product decoder then reconstructs the network structure.
  2. The Negative Sample Generator ($G_{NSG}$): This component acts as a "competitor" by generating fake node attributes from Gaussian noise, creating a challenging adversarial environment.
  3. The Mutual Information Discriminator ($D$): Rather than evaluating embeddings directly, this discriminator assesses the Mutual Information (MI) between inputs and outputs. It learns to distinguish between the "real" mapping produced by the encoder and the "fake" mapping from the negative generator.
The AMIL framework architecture, illustrating the interaction between the Autoencoder, Mutual Information Discriminator, and Negative Sample Generator.

By maximizing the mutual information between attributes and embeddings while competing against the generator, the encoder is forced to learn a highly representative mapping that captures the essential semantic features of the network.

3. Experimental Results

The authors evaluated AMIL on seven benchmark datasets, including webpage networks (WebKB) and citation networks (Cora, Citeseer, Pubmed).

3.1 Performance Metrics

AMIL demonstrated significant improvements over state-of-the-art baselines like ARVGE and DGI:

  • Node Classification: AMIL achieved an average performance boost of 13.46% using LibSVM and 10.08% using LibLINEAR compared to ARVGE.
  • Node Clustering: The model improved Accuracy by 10.45% and Normalized Mutual Information (NMI) by 18.91% over DGI.

3.2 Visualization

The qualitative impact of the AMIL framework is most evident in low-dimensional visualizations. Using t-SNE, the authors compared the embeddings generated by AMIL against those from ARVGE.

t-SNE visualization comparing node embeddings from ARVGE (left) and AMIL (right), showing clearer cluster separation in AMIL.

As shown in the visualization, AMIL produces much tighter and more distinct clusters, whereas previous methods often resulted in interweaved categories.

4. Conclusion

AMIL represents a paradigm shift in adversarial network embedding by prioritizing the quality of the mapping mechanism over the final distribution of the latent space. By using mutual information as a proxy for information preservation and introducing a strong GAN-based competitor, the framework achieves superior robustness and discriminative power. Future work may address the computational complexity of training three adversarial components and the scalability of the model to massive, dynamic networks.

Summary updated

Abstract

Network embedding which is to learn a low dimensional representation of nodes in a network has been used in many network analysis tasks. Some network embedding methods, including those based on generative adversarial networks (GAN) (a promising deep learning technique), have been proposed recently. Existing GAN-based methods typically use GAN to learn a Gaussian distribution as a priori for network embedding. However, this strategy makes it difficult to distinguish the node representation from Gaussian distribution. Moreover, it does not make full use of the essential advantage of GAN (that is to adversarially learn the representation mechanism rather than the representation itself), leading to compromised performance of the method. To address this problem, we propose to use the adversarial idea on the representation mechanism, i.e. on the encoding mechanism under the framework of autoencoder. Specifically, we use the mutual information between node attributes and embedding as a reasonable alternative of this encoding mechanism (which is much easier to track). Additionally, we introduce another mapping mechanism (which is based on GAN) as a competitor into the adversarial learning system. A range of empirical results demonstrate the effectiveness of the proposed approach.
BibTeX
@inproceedings{he2020adversarial,
  author = {Dongxiao He and Lu Zhai and Zhigang Li and Di Jin and Liang Yang and Yuxiao Huang and Philip S. Yu},
  title = {{Adversarial Mutual Information Learning for Network Embedding}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/459},
}
Figure