Edge-Featured Graph Attention Network

Jun Chen, Haopeng Chen

Abstract

Lots of neural network architectures have been proposed to deal with learning tasks on graph-structured data. However, most of these models concentrate on only node features during the learning process. The edge features, which usually play a similarly important role as the nodes, are often ignored or simplified by these models. In this paper, we present edge-featured graph attention networks, namely EGATs, to extend the use of graph neural networks to those tasks learning on graphs with both node and edge features. These models can be regarded as extensions of graph attention networks (GATs). By reforming the model structure and the learning process, the new models can accept node and edge features as inputs, incorporate the edge information into feature representations, and iterate both node and edge features in a parallel but mutual way. The results demonstrate that our work is highly competitive against other node classification approaches, and can be well applied in edge-featured graph learning tasks.

Introduction

Graph Neural Networks (GNNs) have become a standard for learning on graph-structured data. However, a significant limitation of popular architectures like Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs) is their primary focus on node features. In many real-world applications, such as financial trading networks, the edges (representing transactions) carry information as critical as the nodes (representing accounts). Transaction amounts, frequencies, and timestamps are often more indicative of behavior than static user profiles.

To address this, the Edge-Featured Graph Attention Network (EGAT) is proposed. EGAT elevates edge features to "first-class citizens," allowing for a symmetrical learning process where both node and edge representations are updated iteratively and in parallel.

The EGAT Architecture

The core of the EGAT model is the symmetrical EGAT layer. Unlike standard GAT layers that only output updated node features, an EGAT layer processes both node features ($H$) and edge features ($E$) to produce updated representations ($H'$) and ($E'$).

The structure of an EGAT layer and the overall multi-layer architecture.

Node and Edge Attention Blocks

The architecture consists of two main components:

  1. Node Attention Block: This extends the standard GAT mechanism by incorporating edge features into the attention weight calculation between nodes. This ensures that the relationship strength is informed by the specific attributes of the connection.
  2. Edge Attention Block: This novel component updates edge features by treating them as nodes in a transformed graph. By conceptually inverting the graph—where edges become nodes and connections represent shared vertices—the model can apply attention mechanisms to aggregate information from neighboring edges.

Mapping Transformation and Merge Strategy

To facilitate the edge updates, the model employs a mapping transformation. This process uses a mapping matrix ($M_E$) to project edge features into a structure that the attention mechanism can process, effectively creating a "dual graph" representation.

Illustration of the graph mapping transformation used to process edge features.

Furthermore, the architecture utilizes a Multi-scale Merge Strategy. Instead of relying solely on the final layer's output, EGAT concatenates the intermediate features from every layer ($H_m^1, H_m^2, \dots, H_m^L$). This allows the final classifier to leverage patterns captured at different topological scales.

Experimental Evaluation

The authors evaluated EGAT across two distinct types of datasets:

  • Node-Sensitive Benchmarks: Standard citation networks (Cora, Citeseer, Pubmed) where node features are dominant.
  • Edge-Sensitive Financial Data: Two new datasets, Trade-B (binary) and Trade-M (ternary), representing real-world trading records where edges contain multi-dimensional transaction data.

Key Results

EGAT demonstrated exceptional performance on edge-sensitive tasks. On the Trade-B dataset, EGAT achieved 92.0% accuracy, significantly outperforming the standard GAT's 65.0%. This highlights the importance of explicitly modeling edge features when they contain high-signal information. Furthermore, EGAT remained competitive on standard node-centric benchmarks, proving its versatility.

Conclusion

EGAT introduces a robust framework for incorporating edge features into graph learning. By maintaining symmetry between node and edge updates and utilizing a dual-graph transformation, the model captures complex relational patterns often missed by traditional GNNs. Future work will focus on optimizing computational efficiency for dense graphs and extending the architecture to support directed and multi-graphs.

Summary updated

Abstract

Lots of neural network architectures have been proposed to deal with learning tasks on graph-structured data. However, most of these models concentrate on only node features during the learning process. The edge features, which usually play a similarly important role as the nodes, are often ignored or simplified by these models. In this paper, we present edge-featured graph attention networks, namely EGATs, to extend the use of graph neural networks to those tasks learning on graphs with both node and edge features. These models can be regarded as extensions of graph attention networks (GATs). By reforming the model structure and the learning process, the new models can accept node and edge features as inputs, incorporate the edge information into feature representations, and iterate both node and edge features in a parallel but mutual way. The results demonstrate that our work is highly competitive against other node classification approaches, and can be well applied in edge-featured graph learning tasks.
BibTeX
@misc{chen2021edge,
  author = {Jun Chen and Haopeng Chen},
  title = {{Edge-Featured Graph Attention Network}},
  year = {2021},
  eprint = {2101.07671},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.07671},
}
Figure