TL;DR
This paper introduces LP-TGNN, a novel framework that bridges the gap between graph neural networks (GNNs) and domain adaptation techniques. By employing a tensor-based architecture to capture holistic graph information and a label propagation mechanism to reduce domain discrepancies, LP-TGNN effectively adapts GNNs to new, unlabeled graph datasets, significantly outperforming existing methods.
What Problem Does This Paper Address?
Graph Neural Networks (GNNs) have achieved remarkable success in graph classification tasks. However, their performance heavily relies on extensive labeled data within a single domain. This creates a significant bottleneck, as acquiring labeled graph data (e.g., for chemical compounds or social networks) is often prohibitively expensive and time-consuming. When GNNs are trained on one domain and applied to another with different data distributions (domain shift), their performance degrades substantially. Traditional domain adaptation methods, while successful in domains like computer vision, face challenges when applied to graphs due to the inherent complexity and structure of graph data. Specifically, two key problems arise: 1) effectively capturing holistic graph representations that go beyond local connectivity, and 2) reducing domain discrepancies in a way that is suitable for structured graph data.
Methods
The proposed Label-Propagation Tensor Graph Neural Network (LP-TGNN) framework addresses these challenges through two main components:
- Tensor Graph Neural Network (TGNN) Encoder: This backbone encoder is designed to extract rich, holistic graph representations. It comprises two branches:
- Graph Convolutional Branch: This branch leverages the standard message-passing framework of GCNs, stacking multiple layers to enhance its expressive power and capture local neighborhood information.
- Topological Learning Branch: This branch utilizes Persistent Homology (PH) to extract topological features. It converts persistence diagrams into persistence images (PIs) and then combines these PIs from multiple filtration functions into a tensor, capturing multi-modal topological information.
- Tensor Transformation Layer (TTL): A crucial component that seamlessly integrates the outputs from both branches. TTL preserves tensor structures and low-rank properties, allowing for the combination of diverse graph attributes while maintaining discriminative features. This layer is key to creating a unified, expressive representation.
- Label Propagation for Domain Adaptation: Instead of traditional adversarial methods that can discard discriminative information, LP-TGNN employs label propagation to reduce domain discrepancy. For a target graph, its "neighbors" are defined as source graphs with the same pseudo-label predicted by an MLP classifier. This approach leverages consistency regularization, encouraging neighboring data points to have similar labels, which is particularly effective for graphs where defining explicit distance metrics is challenging.
The overall LP-TGNN model is trained jointly using a supervised loss on labeled source data and a consistency regularization loss on unlabeled target data, guided by the label propagation mechanism.
Datasets
The paper evaluates LP-TGNN on several real-world graph classification benchmarks, adapted for domain adaptation settings. These include:
- MUTAG: Predicts mutagenicity of nitroaromatic compounds.
- Mutagenicity: Predicts mutagenicity of molecular structures.
- PROTEINS: Classifies proteins as enzymes or not.
- DD: Classifies proteins as enzymes or not.
- COX2: Chemical compounds related to cyclooxygenase-2 inhibitors.
- BZR: Chemical compounds related to the benzodiazepine receptor.
To create domain discrepancies, the Mutagenicity dataset is split into four subsets (M0-M3) based on edge density quartiles.
Key Insights and Ideas
- Holistic Graph Representation is Crucial: The paper strongly emphasizes that effectively capturing graph structure requires more than just local connectivity. Combining graph convolutional features with topological information (via Persistent Homology) leads to richer representations.
- Label Propagation is a Natural Fit for Graph Domain Adaptation: Unlike adversarial methods that can struggle with structured data, label propagation, when adapted to define neighbors based on pseudo-labels, proves highly effective in bridging domain gaps for graphs.
- Tensorization Unifies Diverse Features: The Tensor Transformation Layer (TTL) is a key innovation that allows for the seamless integration of different types of graph features (convolutional and topological) into a unified representation space, enhancing the model's overall power.
- Generalizability of the Framework: The LP-TGNN framework is designed to be compatible with general GNNs and domain adaptation techniques, suggesting its potential for broader application beyond the specific TGNN encoder.
Key Results
- State-of-the-Art Performance: LP-TGNN consistently achieves the best performance across various domain adaptation benchmarks, outperforming both domain-specific GNNs and general domain adaptation methods. For instance, on average, LP-TGNN achieves 71.8% accuracy on the PROTEINS, COX2, and BZR datasets.
- Effectiveness of Label Propagation: LP-GIN (a GIN encoder with label propagation) significantly outperforms DM-GIN (GIN with distribution matching), highlighting the advantage of label propagation for graph domain adaptation.
- Ablation Studies Confirm Component Importance: Ablation studies show that both the graph convolutional branch and the topological learning branch are crucial for LP-TGNN's success. Removing either significantly degrades performance. The supervised loss and label propagation regularization are also indispensable.
- Subpopulation Shift Handling: A synthetic study demonstrates LP-TGNN's robustness to significant subpopulation shifts, where it achieves high accuracy even when the distribution of positive and negative samples changes drastically between domains.
Limitations and Future Work
- Computational Complexity: While effective, the tensor-based approach and the integration of multiple components can lead to higher computational costs compared to simpler GNN models.
- Generalizability to Larger Datasets: The paper suggests future work could extend the approach to large-scale datasets and more complex settings.
- Multi-label and Multi-target Adaptation: The current framework focuses on single-label classification. Future research could explore its extension to multi-label classification and multi-target domain adaptation scenarios.
Summary updated