In computer vision and natural language processing, foundation models are the norm. A single model can be trained on millions of images or web pages and then applied to entirely new tasks with minimal tweaking. This is possible because these domains possess universal input primitives: pixels always represent color values, and tokens always map to a shared vocabulary.
In Graph Machine Learning, however, this has been a persistent headache. Most Graph Neural Networks (GNNs) are "one-graph wonders," trained from scratch on specific datasets and incapable of generalizing across different domains. As argued by Haitao Mao et al. in Graph Foundation Models Are Already Here, the field has historically lacked a shared "graph vocabulary" that encodes structural invariance across diverse domains like social networks and molecular biology.
The core issue is feature incompatibility. A node feature vector in a citation network might be a 128-dimensional bag-of-words vector, while a node in a molecular graph might be a 5-dimensional vector encoding atomic properties. Because their feature dimensions and underlying semantic distributions are completely misaligned, a model trained on one graph cannot even ingest the data of another.
An emerging paradigm solves this problem by using natural language as a universal feature space. By translating domain-specific graph attributes into text descriptions and encoding them with Large Language Models (LLMs), researchers are creating a unified semantic space where a single model can process entirely different graphs.
Text as the Universal Intermediary
The core intuition behind this approach is that natural language is flexible enough to describe any domain. Instead of feeding raw, heterogeneous feature vectors into a GNN, we can convert node attributes, edge properties, and tabular data into textual descriptions. For example, an atom node becomes "An oxygen atom with a partial charge of -0.5," while a paper node becomes its title and abstract.
[Heterogeneous Graphs]
├── Citation Graph ──> "Paper on deep learning..." ──┐
├── Molecule Graph ──> "Oxygen atom, charge -0.5" ──┼─> [Frozen LLM Encoder] ─> [Unified Semantic Space]
└── Social Network ──> "User interested in tech..." ──┘
By passing these descriptions through a language model, the disparate feature spaces are projected into a shared, continuous embedding space. As demonstrated by Hao Liu et al. in One For All: Towards Training One Graph Model For All Classification Tasks, this textualization allows a single model architecture with a single set of weights to process graphs from completely different domains within a consistent mathematical framework.
This mechanism effectively decouples the model from fixed input dimensions and domain-specific classification layers. In UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs, Bryan Hooi et al. show that representing both node attributes and task labels as text descriptions allows a single model to handle heterogeneous data types simultaneously, enabling zero-shot transfer to entirely unseen graph domains.
Aligning Structure and Semantics
Unifying features is only half the battle; graphs also contain structural topology. To build an effective graph model, the unified text embeddings must be aligned with the graph's connectivity patterns.
What if a graph has no text attributes to begin with? To bridge this gap, Zehong Wang et al. introduced a framework called TANS (Topology-Aware Node description Synthesis) in Can LLMs Convert Graphs to Text-Attributed Graphs?. TANS uses an LLM to "read" the raw structural roles of nodes in a graph and write textual descriptions for them. By encoding structural roles directly into text, the authors convert purely structural graphs into Text-Attributed Graphs (TAGs), resolving feature misalignment before any GNN training begins.
[Raw Graph Structure] ──> [LLM (TANS)] ──> "This node acts as a central hub connecting three distinct communities..."
When text attributes are available, models can learn universal text-to-structure alignment principles. In UniGLM: Training One Unified Language Model for Text-Attributed Graphs Embedding, Yi Fang et al. train a language model across multiple diverse TAGs simultaneously. Because the text attributes share a global semantic space that transcends individual graph topologies, the model learns how language patterns correlate with connectivity patterns.
This shared conceptual framework allows for the transfer of structural intuition across highly divergent graph types. For instance, Yide Qiu et al. demonstrate in One for All: Universal Topological Primitive Transfer for Graph Structure Learning that aligning structural features with natural language descriptions creates a domain-agnostic space that facilitates the transfer of topological primitives, even when the underlying networks have fundamentally different geometries.
Scaling to Graph Foundation Models
Once graph data is mapped to a unified text space, it becomes possible to train general-purpose Graph Foundation Models (GFMs) using diverse datasets.
- Multi-Task Integration: To handle different task levels (nodes, edges, and entire graphs) within a single framework, Zehong Wang et al. developed task-trees in Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees. By projecting text-based attributes into a shared embedding space, their model (GIT) successfully generalizes across more than 30 diverse datasets.
- Ingesting External Data: This text-space alignment also allows models to leverage external, unlabeled "wild" data. In How to use Graph Data in the Wild to Help Graph Anomaly Detection?, Yuxuan Cao et al. convert tabular features from heterogeneous external graphs into text. This unified semantic space allows them to integrate diverse datasets to learn a more robust representation of normal versus anomalous behavior.
- Sequence-Based Architectures: The unification even extends to standard Transformer backbones. In Cross-Domain Pre-training of Transformers on Text-Attributed Graphs via Random Walks, Ziyuan Tang et al. represent graph nodes as collections of text-attributed random walks. This allows a standard Transformer to process non-sequential, multi-domain graph data and successfully transfer structural understanding from citation networks to molecular structures.
The Balance Between Text and Topology
While text-space unification provides an elegant solution to feature heterogeneity, it introduces new design challenges. The success of a graph foundation model depends heavily on how it balances language modeling with structural learning.
In Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights, Zhikai Chen et al. evaluated over 20 datasets across various text-space GFM training paradigms. The authors revealed a crucial caveat: while LLMs provide a powerful universal language for graphs, the model's performance depends heavily on balancing high-quality text embeddings with task-specific structural inductive biases. Relying too much on text can cause the model to ignore critical topological nuances.
Furthermore, the choice of self-supervised learning paradigm matters when operating in this unified space. To systematically evaluate this, Zhigang Hua et al. introduced GSTBench: A Benchmark Study on the Transferability of Graph Self-Supervised Learning. By using a frozen LLM to resolve feature heterogeneity across a massive 100-million-node graph, they discovered that generative masked autoencoders are significantly more robust and transferable across different graph domains than popular contrastive learning methods.
By treating natural language as the universal interface, the graph machine learning community is moving away from isolated, task-specific architectures. The shared semantic space of language acts as a translation layer, allowing models to finally look past surface-level feature dimensions and focus on learning the universal laws of graph structure.
Key Foundational Work
Position: Graph Foundation Models Are Already Here (2024) — contributed 2 concept elements to this direction.
One For All: Towards Training One Graph Model For All Classification Tasks (2024) — contributed 2 concept elements to this direction.
UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs (2025) — contributed 2 concept elements to this direction.