A Multi-Task Representation Learning Architecture for Enhanced Graph Classification

Yu Xie, Maoguo Gong, Yuan Gao, A. K. Qin, Xiaolong FanFrontiers in Neuroscience 2020 · DOI

Abstract: Graph-structured data, composed of nodes and edges, are ubiquitous in fields like chemistry and biology. Effectively extracting inherent features from these non-Euclidean structures is crucial for determining functions or traits. While Graph Neural Networks (GNNs) have gained popularity for graph classification, most existing strategies follow a single-task learning framework that aggregates node-level representations into a global vector, often neglecting available node-level labels. This paper proposes a Multi-Task Representation Learning (MTRL) architecture that couples graph classification with supervised node classification. By enforcing node-level representations to utilize node labels, the model enhances graph-level feature learning in an end-to-end manner. Experimental results on benchmark datasets show that MTRL significantly outperforms various single-task GNN methods.

Introduction

In the domain of Graph Neural Networks (GNNs), models are typically designed as "specialists" for specific tasks. For graph-level classification—such as determining the toxicity of a molecule—standard models aggregate node features into a single global representation. However, this process often ignores individual node labels (e.g., specific atom types in a molecule) during training.

The authors argue that this single-task approach is inefficient. In bioinformatics and chemistry, the global category of a graph is often deeply intertwined with the specific properties of its constituent nodes. By ignoring node labels, existing GNNs fail to fully exploit available data, leading to limited representational capacity and a higher risk of overfitting. The MTRL architecture addresses this by learning fine-grained node features while simultaneously capturing the macro-structure of the entire graph.

The MTRL Architecture

The core of the Multi-Task Representation Learning (MTRL) framework is hard parameter sharing. The model utilizes the same underlying GNN layers to extract features for both node-level and graph-level tasks.

Schematic depiction of the MTRL architecture, showing the flow from input graph through shared aggregation layers to dual output heads for node and graph classification.

As shown in Figure 1, the architecture consists of three main components:

  1. Shared Backbone: The model processes the graph through several aggregation layers (such as GIN or GraphSAGE) to generate node embeddings.
  2. Dual Heads:
    • Node Classification Head: Uses node embeddings from the final or intermediate layers to predict labels for individual nodes.
  3. Graph Classification Head: Employs a "READOUT" function (e.g., summation or hierarchical pooling) to aggregate node embeddings into a global graph-level representation.
  4. Joint Loss Function: The model is trained end-to-end using a weighted sum of losses: $$\mathcal{L}_{MTRL} = \mathcal{L}_{Graph} + \alpha \cdot \mathcal{L}_{Node}$$ The hyperparameter $\alpha$ balances the importance of the two tasks.

The researchers integrated this framework into two state-of-the-art GNNs: Graph Isomorphism Network (GIN) and DIFFPOOL. In MT-GIN, features from all layers are concatenated to capture multi-scale information. In MT-DIFFPOOL, node features are extracted before pooling layers reduce the node count, ensuring sufficient data for the node classification task.

Experimental Evaluation

The framework was evaluated on five bioinformatics benchmark datasets: MUTAG (mutagenic compounds), PTC (carcinogenic compounds), ENZYMES (protein tertiary structures), PROTEINS (enzyme classification), and NCI1 (anti-cancer activity screening). These datasets are ideal as they provide both graph-level categories and specific node attributes.

Key Results and Insights

The MTRL-enhanced models (MT-GIN and MT-DIFFPOOL) consistently outperformed their single-task counterparts. On the NCI1 dataset, MT-GIN achieved 82.59% accuracy, compared to 77.12% for the standard GIN.

A major finding is that the node classification task acts as a powerful regularizer. It prevents the model from losing local details while optimizing for global labels, leading to better generalization. Furthermore, while single-task GNNs often show near-random performance on node classification (e.g., ~2% on NCI1), the MTRL framework saw this performance skyrocket to ~80%, proving that the shared backbone learns much more meaningful features.

t-SNE visualizations of node and graph representations across different datasets, demonstrating the distinct clustering achieved by the MTRL framework.

Qualitative analysis using t-SNE plots (Figure 3) confirms that MTRL models produce tighter, more distinct clusters for both nodes and graphs compared to the noisier representations generated by single-task models.

Conclusion and Future Work

The MTRL architecture demonstrates that multi-tasking provides a "win-win" scenario for graph neural networks, where local node understanding directly benefits global graph classification. However, the performance remains sensitive to the loss-weighting hyperparameter $\alpha$, which currently requires manual tuning.

Future research will focus on developing self-adaptive mechanisms to balance task losses automatically. Additionally, exploring other auxiliary tasks like unsupervised link prediction or community detection could extend the framework's utility to datasets where explicit node labels are unavailable.

Summary updated

Abstract

Composed of nodes and edges, graph structured data are organized in the non-Euclidean geometric space and ubiquitous especially in chemical compounds, proteins, etc. They usually contain rich structure information, and how to effectively extract inherent features of them is of great significance on the determination of function or traits in medicine and biology. Recently, there is a growing interest in learning graph-level representations for graph classification. Existing graph classification strategies based on graph neural networks broadly follow a single-task learning framework and manage to learn graph-level representations through aggregating node-level representations. However, they lack the efficient utilization of labels of nodes in a graph. In this paper, we propose a novel multi-task representation learning architecture coupled with the task of supervised node classification for enhanced graph classification. Specifically, the node classification task enforces node-level representations to take full advantage of node labels available in the graph and the graph classification task allows for learning graph-level representations in an end-to-end manner. Experimental results on multiple benchmark datasets demonstrate that the proposed architecture performs significantly better than various single-task graph neural network methods for graph classification.
BibTeX
@inproceedings{xie2020multi,
  author = {Yu Xie and Maoguo Gong and Yuan Gao and A. K. Qin and Xiaolong Fan},
  title = {{A Multi-Task Representation Learning Architecture for Enhanced Graph Classification}},
  booktitle = {Frontiers in Neuroscience},
  year = {2020},
  doi = {10.3389/fnins.2019.01395},
}
Figure