Scaling Semantic Parsers to Unrestricted Natural Language

33 concepts33 papers1 proposal

Semantic parsing is the art of translating the messy, ambiguous structure of human language into a precise, machine-readable logical form. For decades, this process worked remarkably well within narrow, well-defined boundaries. If you build a system exclusively to query a flight database or control a smart calendar, you can map specific words directly to a fixed set of database attributes and operations.

However, when you move from these "closed worlds" to the open web—where vocabulary is infinite, syntax is unpredictable, and schemas span thousands of fluid concepts—traditional semantic parsing breaks down. This tension is known as the domain-to-generalization gap. Bridging this gap requires moving away from rigid, hand-crafted rules toward flexible representations, automated data synthesis, and execution-based feedback loops.

The Brittleness of Closed-World Rules

To understand why open-domain semantic parsing is challenging, we must look at how early systems were constructed. Early natural language interfaces relied on tight, deterministic alignments between syntax and semantics.

As Palmer (1981, 1983) observed, inference-driven semantic analysis systems optimized for narrow technical domains require an exhaustive library of domain-specific logical rules. If a user deviates even slightly from the expected phrasing, the system cannot map the sentence to a valid logical form. Hayes et al. (1985) attempted to address this by adding "syntactic generality" to semantic caseframe parsers, yet the underlying systems remained fundamentally restricted to their predefined semantic scopes. They simply could not handle arbitrary, open-domain prose.

[Traditional Closed-World Parsing]
"Flights to Boston" ───(Rigid Lexicon)───> Flight(Destination="Boston") ───> Success!
"Get me to Beantown" ──(Unseen Phrase)───> [Parser Failure: Unknown Predicate]

[Open-Domain Parsing Goal]
"Get me to Beantown" ──(Flexible Graph/Execution)───> Entity(Boston) ───> Success!

Even as the field shifted toward statistical methods, the reliance on fixed lexicons persisted. For instance, the breakthrough probabilistic Combinatory Categorial Grammar (CCG) approach introduced by Zettlemoyer et al. (2012) automatically induced grammars to navigate linguistic ambiguity. Yet, these models were still optimized for narrow database queries. As Baral (2011) argued, translating natural language into rigid formal representations works well in controlled environments like biology textbooks, but scaling them to the open world requires handling exponentially higher levels of structural variety and vocabulary noise.

Decoupling Meaning from Rigid Logic

If rigid mathematical logic formalisms (like lambda calculus) create a bottleneck, how do we represent meaning across unrestricted domains? The answer lies in adopting more flexible, structural representations that decouple the parser from a specific database schema.

One powerful alternative is to represent meaning as a directed graph rather than a linear string of logic. Le et al. (2012) introduced DeSCoG (Dependency-based Semantic Composition using Graphs), a framework that leverages off-the-shelf dependency parsers to map natural language directly to semantic graphs. Because graphs can dynamically link entities and relations without requiring a strict, predefined vocabulary order, they allow parsers to generalize to open-domain text like news and literature.

For task-oriented applications, such as digital assistants, requests are often nested and compositional (e.g., "Set a reminder for my meeting when I arrive at the airport"). To handle this without exploding the rule space, Gupta et al. (2018) proposed TOP (Task Oriented Parsing). Instead of flat slot-filling lists, TOP models queries as hierarchical grammar trees. This structural flexibility allows parsing models to scale across entirely different domains—like shifting from navigation to shopping—by treating domain adaptation as a tree-generation problem rather than a slot-matching problem.

Bootstrapping the Open World

An open-domain parser must understand thousands of distinct predicates and entities. Hand-annotating a dataset of this scale is economically and logistically impossible. Researchers have bypassed this data bottleneck using two primary strategies: unsupervised knowledge graph alignment and retrieve-and-edit data synthesis.

Unsupervised Graph Crawling

Instead of asking humans to label text, why not use existing large-scale knowledge bases to label the text for us? Heck et al. (2013) developed a web-scale unsupervised semantic parsing framework by crawling knowledge graphs like Freebase. By automatically aligning text documents with the facts stored in the graph, the system bootstrapped a semantic parser without a single manual label. The challenge here shifts from data collection to noise management, as the system must learn to filter out incorrect alignments across thousands of diverse domains simultaneously.

Retrieve-and-Edit Synthesis

When adapting a parser to a brand-new database or domain, we often lack parallel training data (natural language queries paired with logical forms). Awasthi et al. (2022) introduced REFILL, a framework that synthesizes textually diverse training data. Instead of generating synthetic queries from scratch—which often results in robotic, predictable language—REFILL retrieves real human queries from an existing domain and edits them to fit the schema of the new target domain. This preserves the natural variety, idioms, and complexity of human speech, giving the parser the linguistic robustness it needs to survive in an unrestricted environment.

[REFILL Framework Intuition]
1. Source Domain Query:  "Show me the capacity of the stadium built in 1995."
2. Retrieve & Mask:      "Show me the [attribute] of the [entity] [relation] [value]."
3. Target Domain Edit:    "Show me the salary of the employee hired in 2010."

Evaluating by Execution

In a narrow domain, you can evaluate a semantic parser by checking if its predicted logical form exactly matches a gold-standard string. In an open-domain setting, this metric fails. There are often dozens of syntactically distinct but semantically identical ways to write a complex query or program.

To solve this, the field has converged on executable semantic parsing. As Liang (2016) explains, we should treat the logical form as a program to be executed. The parser is successful not if its code looks like the ground truth, but if executing that code yields the correct answer from the environment or knowledge base.

To scale this paradigm to complex, multi-turn dialogues with intricate logical dependencies, He et al. (2023) developed the DFEE (DataFlow Execution and Evaluation) toolkit. DFEE shifts evaluation away from brittle surface-level code matching toward a holistic "Execution Accuracy" metric. By focusing on the execution side-effects, developers can build and debug parsers that handle complex temporal reasoning and multi-domain tasks without being penalized for valid syntactic variations.

Ultimately, the transition from domain-restricted to open-domain semantic parsing is a transition from rigid translation to flexible execution. By decoupling meaning from strict lexicons, leveraging automated data synthesis, and grounding evaluation in execution, modern parsers can finally begin to process the full breadth of human language.

Key Foundational Work

A Case for Rule-Driven Semantic Processing (1981) — contributed 1 concept elements to this direction.
Generating Hierarchical Semantic Networks Froma Natural Language Discourse (1975) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_open_domain_semantic_parsing_5c7ac888,
  title = {open-domain semantic parsing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/open-domain-semantic-parsing/rdr_5c7ac888272a4e8ca40ca4adf039e754},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_open_domain_semantic_parsing_5c7ac888_v1,
  title = {open-domain semantic parsing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/open-domain-semantic-parsing/rdr_5c7ac888272a4e8ca40ca4adf039e754/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake