When humans produce language, we rarely think in fully formed, grammatically perfect sentences from the outset. Instead, we begin with an abstract communicative goal—a collection of facts, a sequence of events, or a web of concepts—and then translate that internal representation into structured, linear text.
In Natural Language Generation (NLG), this process is split into two main stages: content planning (deciding what to say) and surface realization (deciding how to say it grammatically and fluently). While content planning organizes raw data or semantic concepts, surface realization bridges the gap between these abstract internal representations and human-readable text.
The Content-Realization Gap
Why is surface realization a distinct and difficult problem? The core challenge lies in a fundamental structural mismatch: semantic representations are typically non-linear, unordered, and abstract, whereas natural language is linear, highly ordered, and morphologically complex.
Consider an abstract representation like an Abstract Meaning Representation (AMR) graph, a database table, or a set of predicate-argument frames. These structures capture the core "meaning" of an utterance but omit crucial linguistic details:
- Word Order: Graphs and tables do not inherently specify which words should come first.
- Morphology and Inflection: Abstract representations usually store base lemmas (e.g.,
go) rather than fully inflected verbs (went,goes,is going), which depend on tense, aspect, and agreement. - Syntactic Structure: A single semantic idea can be expressed through many different syntactic forms. For example, a descriptive attribute can be realized as an adjective, a prepositional phrase, or a relative clause.
As highlighted by John A. Bateman et al., a significant gap exists between the high-level propositions produced by text planners and the low-level requirements of complex grammars. If a system simply selects the right content but lacks a sophisticated surface realization component, the final output often degrades into an incoherent collection of keywords rather than fluent sentences, as noted by Fei Liu et al..
Moreover, Bernd Bohnet et al. distinguish between shallow realization—which starts from a syntactic tree that already defines the grammatical structure—and deep generation. Deep generation requires transforming abstract semantic concepts and their relationships into a valid syntax entirely from scratch, making it a much steeper technical hurdle.
The Mechanics of Symbolic and Rule-Based Realization
Early approaches to surface realization relied on explicit linguistic frameworks to map meaning to form. To understand why these systems work, imagine a translator who uses a strict set of grammatical blueprints to convert a conceptual map into a sentence.
One foundational paradigm is Systemic Functional Grammar, which views language as a series of choices determined by the context and communicative goals. In an early text-production system outlined by Christian M. I. M. Matthiessen, abstract knowledge representation was integrated with "choice experts" that mapped conceptual roles directly to syntactic structures. Similarly, Kathleen McKeown et al. leveraged Functional Unification Grammar (FUG) to state global constraints—such as information focus and voice (active vs. passive)—separately from rigid syntactic rules, allowing the generator to flexibly choose the most appropriate linguistic expression for a given context.
However, a common engineering bottleneck in these systems was the friction between application-specific data structures and general linguistic grammars. To resolve this, Stephan Busemann demonstrated the utility of decoupling the two layers using a modular set of "Pattern-Action" rules. By translating application semantics into an intermediate linguistic representation before applying the grammar, developers could avoid opaque, unmaintainable code.
When properly tuned, these symbolic mapping rules are remarkably powerful. As Charles B. Callaway demonstrated, failures in symbolic generation pipelines are frequently caused by incomplete transformation rules between syntax and semantics rather than limitations of the underlying grammar itself; refining this mapping interface can unlock massive latent linguistic coverage.
Modern Hybrid and Controlled Realization
With the advent of machine learning and large language models (LLMs), one might assume that the surface realization bottleneck would disappear. However, modern research shows that even when a model is excellent at generating text, the disconnect between abstract planning and surface expression persists. For instance, Heng Gong et al. observed that optimizing structured content planning in data-to-text systems does not automatically improve linguistic fluency or word-overlap metrics, confirming that surface realization remains a distinct performance bottleneck.
To address this in the neural era, researchers have developed hybrid architectures that break the realization process into manageable, specialized sub-tasks.
Instead of forcing a single neural network to map an abstract semantic graph directly to a finished sentence, systems often divide the labor. For example, Farhood Farahnak et al. designed a surface realizer that combines a Transformer encoder-decoder with a pointer network specifically tasked with word ordering, paired with a separate feed-forward network dedicated entirely to morphological inflection. By isolating the structural problem (ordering) from the grammatical problem (inflection), the system achieves higher precision.
[Abstract Semantic Graph]
│
▼
┌──────────────────────────────────┐
│ Transformer + Pointer Net │ ──► Resolves Word Order
└──────────────────────────────────┘
│
▼
┌──────────────────────────────────┐
│ Feed-Forward Network │ ──► Resolves Morphological Inflection
└──────────────────────────────────┘
│
▼
[Fluent, Grammatical Sentence]
Another major focus of modern surface realization is controllability. When generating text from abstract semantic graphs like AMRs, standard language models can struggle to balance natural fluency with precise structural constraints. To give models explicit "knobs and dials," Kathleen McKeown et al. developed a method to augment semantic graphs with fine-grained syntactic tags. By explicitly feeding tags for verb tense, voice, and formatting into a pretrained language model alongside the semantic meaning, they achieved precise control over the final phrasing without sacrificing the model's natural fluency.
Ultimately, surface realization works because it treats language generation not as a single monolithic prediction, but as a structured translation from a multi-dimensional web of meaning into a linear, grammatically compliant sequence of words.
Key Foundational Work
The machine realization of the periphrasing system and the results of the experiment (1969) — contributed 2 concept elements to this direction.
SGS: A System for Mechanical Generation of Japanese Sentences (1980) — contributed 1 concept elements to this direction.