GraphShaper: Geometry-aware Alignment for Improving Transfer Learning in Text-Attributed Graphs

Heng Zhang, Tianyi Zhang, Yuling Shi, Xiaodong Gu, Yaomin Shen, Haochen You, Zijian Zhang, Yilei Yuan, Jin HuangarXiv 2025 · DOI · arXiv

TL;DR

Current graph foundation models struggle at structural boundaries due to a one-size-fits-all Euclidean encoding. GraphShaper introduces a novel framework that leverages multiple geometric spaces (Euclidean, hyperbolic, spherical) to represent graph structures adaptively, significantly improving transfer learning performance, especially in zero-shot settings.

What Problem Does This Paper Address?

Text-Attributed Graphs (TAGs) combine the rich information from text with the relational structure of graphs, making them powerful for various downstream tasks. Large Language Models (LLMs) have been integrated with graph learning to enhance representation learning, often by unifying graph and text modalities into a shared embedding space. However, a critical limitation has emerged: these models exhibit significant performance degradation at "structural boundaries" – areas where different topological patterns (like trees and cycles) converge. This is because current methods assume all graph structures can be effectively represented in a single Euclidean space. In reality, hierarchical structures (like trees) are better suited to hyperbolic geometry, while cyclic structures benefit from spherical geometry. Forcing these diverse structures into a single Euclidean space leads to distortions and compromises the model's ability to generalize, particularly in transfer learning scenarios.

Methods

GraphShaper tackles this problem by introducing a geometry-aware alignment framework that moves beyond a single Euclidean space. The core innovation lies in its multi-geometric specialization:

  • Geometry-aware Graph Encoding: Instead of a uniform encoder, GraphShaper uses specialized "expert" networks for Euclidean, hyperbolic, and spherical geometries. These experts process local graph structures (e.g., k-hop neighborhoods) and encode them within their respective optimal geometric spaces. This is achieved using Riemannian graph convolutions tailored to each geometry.
  • Adaptive Geometric Fusion: At structural boundaries, where conflicting geometric constraints arise, GraphShaper employs a mixture-of-experts architecture. A learnable gating network dynamically computes fusion weights, allowing the model to adaptively combine the representations from the different geometric experts based on the local structural context. This prevents forced trade-offs and preserves structural integrity.
  • Geometry-adaptive Alignment: The framework aligns the geometry-aware graph representations with text embeddings using a contrastive learning objective. This alignment is designed to be robust to geometric transformations.
  • Model Adaptation: For zero-shot and few-shot learning, GraphShaper utilizes prompt tuning and learned prompt features, allowing it to adapt to new tasks and domains without extensive retraining of the core model.

Datasets

The paper evaluates GraphShaper on seven text-attributed graph datasets:

  • Academic: Cora, Pubmed, ArXiv
  • E-commerce: Children, History, Photo, Products

These datasets cover diverse structural patterns and semantic contexts, allowing for a comprehensive evaluation of the model's generalization capabilities.

Key Insights and Ideas

  • The "Structural Boundary Problem" is Geometric: The paper convincingly demonstrates that performance drops at structural boundaries are not due to model capacity but a fundamental geometric mismatch in current encoding strategies.
  • Geometry is Key to Transfer Learning: By respecting the intrinsic geometric properties of different graph structures, GraphShaper achieves significantly better transfer learning performance, especially in zero-shot settings, where models must generalize to unseen tasks or domains.
  • Adaptive Fusion is Crucial: The ability to dynamically fuse representations from different geometric experts is what allows GraphShaper to handle the complexity of structural boundaries effectively, rather than relying on a single, potentially suboptimal, geometric representation.
  • Diversity Promotes Specialization: The paper highlights the importance of maximizing "weight diversity" among the geometric experts to ensure they develop distinct and complementary capabilities.

Key Results

GraphShaper demonstrates substantial improvements across various tasks and settings:

  • Zero-Shot Node Classification: Achieved accuracy gains of 9.47% on citation networks and 7.63% on social networks compared to strong baselines. On specific datasets like Pubmed, it reached 89.70% accuracy.
  • Cross-Dataset Generalization: Showed significant improvements in zero-shot node classification across multi-dataset transfer settings, outperforming baselines like RiemannGFM by up to 5.7%.
  • Link Prediction: Achieved consistent improvements in AUC scores, with 70.66% AUC on Pubmed.
  • Node Description Quality: Demonstrated superior performance in node description tasks, with SBERT scores exceeding 90% on Cora and Products.

Limitations and Future Work

  • Computational Cost: While GraphShaper offers improved performance, the use of multiple geometric experts and Riemannian convolutions might introduce higher computational overhead compared to single-space models. The paper mentions efficiency trade-offs in Figure 6.
  • Hyperparameter Sensitivity: The performance is shown to be sensitive to hyperparameter choices, particularly curvature parameters and diversity loss weights (as explored in Section 5.6). Careful calibration is required.
  • Future Work: The authors suggest exploring more sophisticated geometric spaces and adaptive fusion mechanisms. Further investigation into the interpretability of geometric representations and their impact on downstream tasks is also a promising direction. The framework's potential for few-shot learning and its application to a wider range of graph-related tasks could also be explored.

Summary updated

Abstract

Graph foundation models represent a transformative paradigm for learning transferable representations across diverse graph domains. Recent methods leverage large language models to unify graph and text modalities into a shared representation space using contrastive learning. However, systematic evaluations reveal significant performance degradation at structural boundaries where distinct topological patterns converge, with accuracy losses exceeding 20 percentage points. This issue arises from a key limitation: current methods assume all graph structures can be encoded within a single Euclidean space. In reality, tree structures require hyperbolic geometry to preserve hierarchical branching, while cyclic patterns depend on spherical geometry for closure properties. At structural boundaries, nodes experience conflicting geometric constraints that uniform encoding spaces cannot resolve. This raises a crucial challenge: \textbf{Can alignment frameworks be designed to respect the intrinsic geometric diversity of graph structures?} We introduce \textbf{GraphShaper}, a geometry-aware framework that enhances graph encoding through multi-geometric specialization. Our approach employs expert networks tailored to different geometric spaces, dynamically computing fusion weights to adaptively integrate geometric properties based on local structural characteristics. This adaptive fusion preserves structural integrity before alignment with text embeddings. Extensive experiments demonstrate that GraphShaper achieves 9.47\% accuracy improvements on citation networks and 7.63\% on social networks in zero-shot settings.
BibTeX
@misc{zhang2025graphshaper,
  author = {Heng Zhang and Tianyi Zhang and Yuling Shi and Xiaodong Gu and Yaomin Shen and Haochen You and Zijian Zhang and Yilei Yuan and Jin Huang},
  title = {{GraphShaper: Geometry-aware Alignment for Improving Transfer Learning in Text-Attributed Graphs}},
  year = {2025},
  eprint = {2510.12085},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.12085},
}
Figure