SLFNet: Generating Semantic Logic Forms from Natural Language Using Semantic Probability Graphs

Hao Wu, Fan XuarXiv 2024 · DOI · arXiv

Introduction

Semantic parsing is a critical task in natural language processing, involving the translation of human language into machine-readable Semantic Logic Forms (SLFs). For instance, a command like "Turn on the lights in the kitchen" must be parsed into a structured format that an IoT system can execute. Traditionally, this has been approached using sequence-to-sequence (seq2seq) frameworks. However, these models often struggle with "selection-sensitivity," where the arbitrary order of slots in a logical form during training can confuse the network. Furthermore, standard models frequently fail to capture long-range syntactic dependencies, making it difficult to correctly associate actions with their intended objects in complex, multi-part commands.

To address these challenges, the authors of this paper propose SLFNet, a model that shifts the paradigm from traditional string generation to a "Sequence-to-Slots" task. By utilizing Semantic Probability Graphs to model dependencies between actions, locations, and objects, SLFNet achieves state-of-the-art performance on complex natural language command datasets.

Methodology

SLFNet decomposes the semantic parsing process into three distinct subtasks: identifying the number of commands, filling the predefined slots, and performing logical inference. The architecture is built upon several key innovations:

Sequence-to-Slots Prediction

Instead of generating a flat, serial string, SLFNet predicts values for specific slots: Action, Location, and Object (ALO). This approach bypasses the "order matters" problem entirely, as the model focuses on populating a structured template rather than deciding which part of the command to output first.

Overview of the Sequence-to-Slots task and the resulting semantic logic forms.

Dependency-Fused BiLSTM and Semantic Probability Graphs

The model incorporates syntactic dependency trees as prior knowledge through a Dependency-Fused BiLSTM. This allows the encoder to recognize grammatical relationships between words that may be far apart in a sentence, such as a verb and its distant object.

Furthermore, the relationship between slots is represented as a directed graph. The prediction for a "Location" slot, for example, is conditioned on both the input text and the previously predicted "Action." This treats slot filling as a causal inference problem, where predicting the Action first provides a logical context that simplifies the prediction of subsequent slots.

Multi-Head SLF Attention

A specialized attention mechanism is employed to identify the specific tokens in the source sentence most relevant to a particular slot label, further enhancing the model's precision in complex linguistic environments.

Experimental Evaluation

The model was rigorously evaluated on several benchmarks, most notably ChineseQCI-TS, a dataset of 21,900 natural language commands designed for smart home and IoT interactions. Other datasets used for scalability and performance verification included Okapi, ATIS (Airline Travel Information System), and WikiSQL.

Key Results

The most advanced version of the model, SLFNet+EM+SA (incorporating both the Dependency-Fused BiLSTM and Multi-Head Attention), significantly outperformed all baselines. On the ChineseQCI-TS dataset, it achieved a test accuracy of 79.7%, representing a substantial improvement over Vanilla Seq2seq (54.3%) and even outperforming a BERT-based model with a copy mechanism (68.5%).

Performance comparison of SLFNet variants against baseline models on the ChineseQCI-TS dataset.

Ablation studies confirmed the importance of each component: the SLF Attention mechanism alone improved accuracy by approximately 5%, while the inclusion of dependency information (EM) provided a massive 14% boost. Detailed metrics also revealed that the model is exceptionally strong at identifying specific slots, with an accuracy of 84.6% for Actions and 87.1% for Locations.

Conclusion and Future Work

SLFNet demonstrates that moving away from the sequence bottleneck and incorporating structural syntactic knowledge can dramatically improve the robustness of semantic parsing. By treating slot filling as a hierarchical, causal process, the model becomes more resilient to variations in human phrasing.

However, some limitations remain. The current "ALO" framework is specifically tailored for command-based IoT interfaces and may require adaptation for more deeply nested logical forms. Additionally, the model's performance is tied to the quality of the external dependency parser used during encoding. Future research will focus on applying these semantic probability graphs to even more diverse and complex structure generation problems.

Summary updated

Abstract

Building natural language interfaces typically uses a semantic parser to parse the user's natural language and convert it into structured \textbf{S}emantic \textbf{L}ogic \textbf{F}orms (SLFs). The mainstream approach is to adopt a sequence-to-sequence framework, which requires that natural language commands and SLFs must be represented serially. Since a single natural language may have multiple SLFs or multiple natural language commands may have the same SLF, training a sequence-to-sequence model is sensitive to the choice among them, a phenomenon recorded as "order matters". To solve this problem, we propose a novel neural network, SLFNet, which firstly incorporates dependent syntactic information as prior knowledge and can capture the long-range interactions between contextual information and words. Secondly construct semantic probability graphs to obtain local dependencies between predictor variables. Finally we propose the Multi-Head SLF Attention mechanism to synthesize SLFs from natural language commands based on Sequence-to-Slots. Experiments show that SLFNet achieves state-of-the-art performance on the ChineseQCI-TS and Okapi datasets, and competitive performance on the ATIS dataset.
BibTeX
@misc{wu2024slfnet,
  author = {Hao Wu and Fan Xu},
  title = {{SLFNet: Generating Semantic Logic Forms from Natural Language Using Semantic Probability Graphs}},
  year = {2024},
  eprint = {2403.19936},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.19936},
}
Figure