Abstract
We detail refinements made to Abstract Meaning Representation (AMR) that make the representation more suitable for supporting a situated dialogue system, where a human remotely controls a robot for purposes of search and rescue and reconnaissance. We propose 36 augmented AMRs that capture speech acts, tense and aspect, and spatial information. This linguistic information is vital for representing important distinctions, for example whether the robot has moved, is moving, or will move. We evaluate two existing AMR parsers for their performance on dialogue data. We also outline a model for graph-to-graph conversion, in which output from AMR parsers is converted into our refined AMRs. The design scheme presented here, though task-specific, is extendable for broad coverage of speech acts using AMR in future task-independent work.
Introduction
Abstract Meaning Representation (AMR) is a semantic formalism designed to capture "who is doing what to whom" in a rooted, directed acyclic graph. While effective for broad-coverage news text, standard AMR lacks the nuance required for situated human-robot dialogue, particularly in high-stakes domains like search and rescue. In these contexts, the distinction between "I am moving," "I moved," and "I will move" is critical for operational safety and coordination. Furthermore, standard AMR does not explicitly encode speech acts—the pragmatic intent behind an utterance, such as whether it is a command, a question, or an assertion. This paper details refinements to AMR that incorporate tense, aspect, and speech acts to better support robot navigation and reconnaissance.
Methodology
The proposed framework introduces 36 augmented AMR templates tailored for the search-and-navigation domain. The architecture relies on a multi-layered approach to semantic representation.
Temporal and Aspectual Logic
To address the lack of temporal context in standard AMR, we integrated a temporal logic that situates events relative to the "speech time." Events are categorized as occurring before, during (now), or after the utterance. Additionally, aspectual markers such as :stable, :ongoing, :complete, and :completable track the status of robot tasks, as shown in the annotation schema below.
Speech Act Anchoring and Executability
Utterances are anchored by PropBank rolesets (e.g., command-02, assert-02) to explicitly encode speaker intent. A novel :completable attribute is used to handle underspecified instructions. For instance, a command like "Move forward" without a specified distance is marked :completable -, prompting the dialogue system to request clarification before execution.
System Pipeline
The implementation utilizes a graph-to-graph conversion model. The pipeline begins with robot-directed natural language, which is processed by a standard AMR parser. The resulting abstract representation is then converted into a domain-specific meaning representation suitable for a Dialogue Manager (DM), as illustrated in the system overview below.
Evaluation and Results
The system was evaluated using a Human-Robot Dialogue Corpus derived from "Wizard-of-Oz" experiments. A gold standard set of 137 adjudicated sentences was used for testing.
Baseline performance of existing AMR parsers (JAMR and CAMR) trained on news data was low, with F-scores between 0.33 and 0.40, highlighting a significant domain gap. However, augmenting the training set with just 504 in-domain sentences improved the JAMR parser's F-score to 0.36 and significantly enhanced its ability to identify spatial parameters, such as correctly mapping distances to the :extent role rather than the :ARG1 (the object being moved).
Conclusion
Augmenting AMR with pragmatic and temporal layers provides a robust foundation for human-robot interaction. While parser performance on conversational speech remains a challenge, the use of graph-to-graph conversion and in-domain fine-tuning demonstrates a viable path toward deploying sophisticated NLU in specialized, situated environments. Future work will focus on expanding the training corpus and integrating these representations into natural language generation (NLG) for robot-to-human feedback.
Summary updated