Large Language Models (LLMs) excel at processing sequential text but often struggle with the rigid, symbolic structure of Knowledge Graphs (KGs). Traditionally, researchers used specialized Graph Neural Networks (GNNs) to handle this structured data. However, a growing body of research suggests a simpler, more effective bridge: linearization. By converting structured triplets into natural language sequences, we can leverage the pre-trained linguistic reasoning of transformers to solve complex relational tasks.
The Semantic Gap
Knowledge graphs represent the world as a web of entities and relations (e.g., [Paris] -capital_of-> [France]). While precise, this format is alien to models like BERT or GPT, which are trained on the fluid, contextual nature of human language.
As Tara Safavi and Danai Koutra argue in their review, Relational World Knowledge Representation in Contextual Language Models, the goal of linearization is to transform these symbolic "facts" into a format that fits the transformer's "native" input space. Instead of building a new architecture to understand a graph, we translate the graph into a language the model already speaks.
From Triples to Pseudo-Sentences
The most common approach to linearization is the use of templates to turn RDF triplets into natural language strings. For example, the triplet (Aspirin, treats, Headache) might be linearized as "Aspirin is used for the treatment of headache."
This transformation allows models to process relational data through standard attention mechanisms. In Towards Medical Machine Reading Comprehension with Structural Knowledge and Plain Text, Baotian Hu and colleagues demonstrate that converting medical knowledge into "pseudo-sentences" enables encoders to interpret rigid factual data as if it were plain text, significantly improving performance in specialized domains where general-purpose models often fail.
Other researchers, such as Lingbing Guo in MKGL: Mastery of a Three-Word Language, take this further by treating triplets as a constrained "three-word language." This approach maintains the atomic nature of entities while allowing the LLM to apply its pre-trained grammatical intuition to structured completion and reasoning.
Why Linearization Works
The intuition behind linearization is that transformers are not just sequence processors; they are powerful pattern matchers for semantic relationships. By flattening a graph into a sequence, we provide the model with several advantages:
- Leveraging Pre-trained Priors: Models like RoBERTa have already seen millions of sentences describing relationships. Linearization aligns the input with these learned patterns. Xin Lv and Yankai Lin show in PKGC that using natural language prompts instead of raw data allows models to retrieve latent factual knowledge more effectively.
- Unified Reasoning: Linearization allows for "joint decoding." In DecAF, Donghan Yu and his team demonstrate that by representing a knowledge base as linearized text passages, they can replace brittle, specialized entity-linking tools with standard information retrieval methods.
- Inductive Capability: Unlike traditional graph embeddings, which often struggle with entities not seen during training, text-based representations can handle "open-world" scenarios. Hanwen Zha and Xifeng Yan show that converting reasoning paths into natural language sequences allows BERT to perform inductive relation prediction by interpreting relational chains as contextual evidence.
Scaling and Specialization
Linearization is particularly potent in data-scarce or highly technical fields. In BioLORD, François Remy uses templates to turn ontological descriptions into training data, grounding "opaque" medical terms in their formal definitions. Similarly, Prix-LM by Fangyu Liu uses a causal language modeling objective on linearized triples to transfer factual knowledge from high-resource languages to low-resource ones.
For massive datasets, researchers have developed efficient ways to index these linearizations. Riccardo Pozzi and the authors of ReFactX organize linearized triplets into a prefix-tree structure. This allows an LLM to "search" through 800 million facts during the decoding process, ensuring that the model only generates verified information without the latency of a traditional retrieval-augmented generation (RAG) pipeline.
Conclusion
The shift toward linearization reflects a broader realization in the field: the inherent structure of a graph may be less important for neural reasoning than the semantic accessibility of its facts. By treating knowledge graphs as a specialized form of text, researchers like Qipeng Guo (CycleGT) have shown that we can overcome the data scarcity bottleneck, allowing models to move seamlessly between structured data and natural language.