HePa: Heterogeneous Graph Prompting for All-Level Classification Tasks

Jia Jinghong, Lei Song, Jiaxing Li, Youyong KongAAAI 2025 · DOI

TL;DR

This paper introduces HePa, a novel framework for heterogeneous graph prompting that unifies node, graph, and edge classification tasks under a single pre-training objective. By leveraging a unified prompt template-graph and a learnable "Pre-Prompt" token, HePa enables effective in-context learning for few-shot scenarios, outperforming existing methods.

What Problem Does This Paper Address?

Heterogeneous graphs, which represent complex relationships with multiple node and edge types, are prevalent in real-world applications. While Heterogeneous Graph Neural Networks (HGNNs) are powerful for tasks like classification, they typically require extensive labeled data for supervised training. To mitigate this, the "pretrain-finetune" paradigm was adopted, but it often leads to a knowledge gap between pre-training and downstream tasks. The more recent "pretrain-prompt" paradigm, inspired by NLP, aims to bridge this gap by using learnable prompt vectors. However, existing graph prompting methods often struggle to handle the diverse nature of heterogeneous graphs and achieve all-level classification (node, graph, and edge) simultaneously. Furthermore, fine-tuning randomly initialized prompts in downstream tasks can lead to incomplete task representations. This paper addresses these challenges by proposing a unified prompting framework for heterogeneous graphs that can handle multiple classification levels and improve task-specific representation.

Methods

HePa introduces a novel framework with three key components:

  • Unified Prompt Template-Graph: This component creates a standardized template that incorporates heterogeneous graph information, allowing for joint training across different task levels (node, graph, and edge). This template includes a "target subgraph" representing the data to be classified, a "prompt node" for task information, and "prototype nodes" for category information.
  • Virtual Prototype Specification: To inject class information into the prototype nodes, HePa employs a sophisticated method. For node classification, it calculates prototype features based on weighted averaging of node features within contextual graphs. For edge classification, it considers the minimum distance to either of the two central nodes. For graph classification, it averages features of all nodes in the graph.
  • Pre-Prompt Token: A crucial innovation is the "Pre-Prompt" token, a learnable vector optimized during the upstream pre-training phase. This token is designed to capture and inject task-specific information, allowing the model to adapt to different downstream tasks without requiring extensive fine-tuning of the entire model. The Pre-Prompt is optimized using a binary classification loss on prototype node embeddings.

The overall framework involves an upstream phase for pre-training the HGNN model with the unified template and a downstream phase where the pre-trained model, with frozen parameters, uses the Pre-Prompt token and task-specific templates for in-context learning.

Datasets

The paper evaluates HePa on three benchmark datasets:

  • IMDB: A large dataset containing movie-related metadata, user ratings, and reviews.
  • DBLP: A comprehensive bibliographic database of computer science journals and conference proceedings.
  • ACM: A dataset of scholarly articles published in ACM conferences and journals.

These datasets are used for both training and evaluating node, graph, and edge classification tasks in few-shot scenarios.

Key Insights and Ideas

  • Unifying All-Level Classification: The most significant insight is the successful unification of node, graph, and edge classification tasks within a single prompting framework. This demonstrates the flexibility and power of the proposed template-graph approach.
  • The Power of the Pre-Prompt Token: The introduction of the learnable "Pre-Prompt" token is a key enabler for effective in-context learning. It allows the model to dynamically adapt to specific downstream tasks by encoding task-relevant information, overcoming the limitations of fixed prompts or random initialization.
  • Prototype Features for Class Representation: The paper highlights the importance of carefully designed prototype feature calculation methods for accurately representing class information, especially in heterogeneous graphs. This explicit computation proves more effective than simply using average features.
  • Cross-Task Capability: HePa exhibits promising cross-task capability, performing well even when trained on one task type and evaluated on others, suggesting effective knowledge transfer.

Key Results

HePa demonstrates superior performance across all three classification levels (node, graph, and edge) on the IMDB, DBLP, and ACM datasets, particularly in few-shot scenarios. For instance, on the IMDB dataset, HePa achieves accuracies of 54.53% for node classification, 43.77% for graph classification, and 53.39% for edge classification in a 5-shot setting, outperforming several state-of-the-art baselines. The paper also shows that HePa's performance consistently improves with an increasing number of shots. Ablation studies confirm the necessity of each component, with the Pre-Prompt token and prototype feature calculation being particularly crucial.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while HePa shows good cross-task capability, its performance on tasks it wasn't jointly trained on is still lower than when jointly trained.
  • The current implementation uses R-GCN as the backbone, which might limit its applicability to other HGNN architectures.
  • Future Work:
    • The authors suggest exploring more advanced HGNN backbones to further enhance performance.
  • Investigating more sophisticated methods for injecting task and category information into the prompt template could lead to further improvements.
  • Expanding the framework to handle other graph-based tasks beyond classification, such as link prediction, is a natural next step.

Summary updated

BibTeX
@inproceedings{jinghong2025hepa,
  author = {Jia Jinghong and Lei Song and Jiaxing Li and Youyong Kong},
  title = {{HePa: Heterogeneous Graph Prompting for All-Level Classification Tasks}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i11.33297},
}
Figure