The Impedance Mismatch Between Graphs and Transformers
Knowledge graphs, molecular structures, and interaction networks are inherently non-linear, multi-dimensional structures defined by nodes and edges. Conversely, state-of-the-art Large Language Models (LLMs) are sequence-processing engines. They are optimized to process linear streams of tokens, leveraging self-attention mechanisms to compute dependencies across a one-dimensional context window.
Faced with the challenge of teaching language models to reason over structural data, a naive approach might involve designing hybrid architectures—such as combining Graph Neural Networks (GNNs) with Transformers—or feeding raw adjacency matrices directly into the model. However, these methods often discard the massive, pre-trained linguistic priors that make LLMs powerful in the first place.
Instead of changing the model architecture to fit the graph, a rapidly growing body of research focuses on changing the data format to fit the model. By linearizing or verbalizing graph topologies into natural language sequences, researchers treat structural data as a text-translation or text-completion task. This allows standard, unmodified language models to apply their pre-trained sequence-modeling capabilities to complex relational reasoning.
From Triples to Sentences: The Mechanics of Verbalization
The fundamental unit of many structured graphs is the knowledge triple: a discrete fact composed of a subject, a predicate, and an object (e.g., [National Highway, is in, Australia]). In its raw database format, this tuple lacks the syntactic markers that trigger an LLM's internal semantic representations.
To bridge this gap, early techniques focused on transforming individual triples into fluent "pseudo-sentences" using predefined templates. As demonstrated by Joe Davison et al. in Commonsense Knowledge Mining from Pretrained Models, converting structured triples into natural sentences allows a model to evaluate the validity of a fact using its internal linguistic coherence. Instead of parsing a database entry, the model simply measures the sentence's Pointwise Mutual Information (PMI) or token probability.
Raw Triple: (National Highway, is in, Australia)
Template: [Subject] is located within [Object].
Verbalization: "National Highway is located within Australia."
This simple transformation yields significant practical benefits. For instance, Isaiah Onando Mulang et al. showed in Evaluating the Impact of Knowledge Graph Context on Entity Disambiguation Models that verbalizing 1-hop triples and appending them directly to an input text stream allows models like RoBERTa to resolve highly ambiguous entities without any architectural modifications. Similarly, Baotian Hu et al. applied this concept to specialized domains in Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text, demonstrating that translating medical knowledge graphs into pseudo-sentences allows standard attention mechanisms to seamlessly blend external factual knowledge with plain text context.
Capturing Topology: Serializing Neighborhoods and Paths
While single-triple verbalization works well for isolated facts, real-world graph reasoning often requires understanding broader topology, such as multi-hop paths or local neighborhoods. Linearizing these structures requires moving from simple templates to sequence-mapping strategies that preserve connectivity.
One approach is neighborhood verbalization, which flattens the immediate structural context of a node into a single continuous string. In Better Together: Enhancing Generative Knowledge Graph Completion with Language Models and Neighborhood Information, Alla Chepurova et al. demonstrated that feeding an LLM a serialized text description of a node's local neighborhood significantly boosts its ability to predict missing links.
When the reasoning task requires traversing the graph, researchers linearize specific structural trajectories. For example, George Karypis et al. in GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning extract the shortest paths between query entities and candidate nodes, converting these mathematical paths into natural language strings that serve as grounded context for the model.
To prevent long linear sequences from becoming an uninterpretable "bag of words" for the model, recent work introduces hierarchical formatting. Chen Ling et al. argued in Link Prediction on Textual Edge Graphs that flat string linearization often fails to capture non-linear dependencies. Instead, they structure local graph topologies into hierarchical documents complete with headers and cross-references, explicitly leveraging the pre-trained document-processing capabilities of modern LLMs to preserve structural relationships.
Generative and Symbolic Translation Pipelines
As the scale of graph data grows, hand-crafted templates become a bottleneck. This has led to the development of automated pipelines that use generative models to handle the serialization process itself.
A prominent example is the TEKGEN pipeline introduced by Oshin Agarwal et al. in Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training. TEKGEN processes millions of structured Wikidata triples and generates a massive, fluent natural language corpus (KELM). By converting dry data triples into grammatically rich text, the pipeline allows language models to ingest factual knowledge through standard pre-training objectives, proving that natural language sentences are a more effective vehicle for knowledge injection than raw data strings.
[Graph Subgraph] ──> (Generative Alignment Model) ──> "Fluent Narrative Summary"
Rather than focusing solely on surface-level fluency, other frameworks design specialized symbolic grammars. In Unify Graph Learning with Text: Unleashing LLM Potentials for Session Search, Songhao Wu et al. developed a consistent symbolic textual grammar where nodes and edges are represented as formatted strings. This allows the model to process structural topology via sequence modeling without losing the mathematical precision of the underlying graph. Similarly, Khushbu Agarwal et al. introduced GLaM (GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding), which uses generative subgraph encoding to rewrite technical database labels into human-interpretable descriptions, facilitating smoother semantic alignment during fine-tuning.
Why It Works: Trading Architecture for Representation
The success of graph serialization lies in a fundamental trade-off: it exchanges architectural complexity for data representation.
When a graph is converted into a serialized text sequence, the model's self-attention layers can compute interactions between any pair of nodes or edges in the sequence. In effect, the attention matrix of the Transformer acts as a dynamic, fully connected graph layer where the strength of the edges is determined by both the structural context provided in the text and the model's pre-trained semantic understanding.
As Yintong Huo et al. demonstrated in CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm, representing structured knowledge as natural language text frequently outperforms specialized graph-reasoning architectures on complex reasoning tasks. In his comprehensive survey, Wenbo Shang (A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications) notes that sequential formats like adjacency lists or descriptive sentences successfully bridge the gap between non-Euclidean structures and text-based models. By formatting graphs as text, researchers avoid the need to train graph encoders from scratch, unlocking the vast reservoir of relational and linguistic intuition already embedded within modern language models.