When Do MLPs Excel in Node Classification? An Information-Theoretic Perspective
Lists
Only you see this
Abstract
Recent research has shed light on the competitiveness of MLP-structured methods in node-level tasks. Nevertheless, there remains a gap in our understanding regarding why MLPs perform well and how their performance varies across different datasets. This paper addresses this lacuna by emphasizing mutual information’s pivotal role in MLPs vs. GNNs performance variations. We first introduce a tractable metric to quantify the mutual information between node features and graph structure, based on which we observe different characteristics of various datasets, aligning with empirical results. Subsequently, we present InfoMLP, which optimizes node embeddings’ mutual information with the graph’s structure, i.e., the adjacency matrix. Our info-max objective comprises two sub-objectives: the first focuses on non-parametric reprocessing to identify the optimal graph-augmented node feature matrix that encapsulates the most graph-related information. The second sub-objective aims to enhance mutual information between node embeddings derived from the original node features and those from the graph-augmented features. This integration of message-passing during preprocessing maintains the efficiency of InfoMLP, ensuring it remains as efficient as a standard MLP during both training and testing. We validate the effectiveness of our approach through experiments on real-world datasets of varying scales supplemented by comprehensive ablation studies. Our results affirm our analysis and underscore the success of our innovative approach.
BibTeX
@misc{zhang2024when,
author = {Hengrui Zhang and Qitian Wu and Chenxiao Yang and Junchi Yan and Philip S. Yu},
title = {{When Do MLPs Excel in Node Classification? An Information-Theoretic Perspective}},
year = {2024},
url = {https://openreview.net/forum?id=2dLMPOY0HW},
howpublished = {OpenReview},
note = {Submitted to ICLR 2024},
}Loading related content...