Abstract
This paper presents an approach for learning to translate simple narratives, i.e., texts (sequences of sentences) describing dynamic systems, into coherent sequences of events without the need for labeled training data. Our approach incorporates domain knowledge in the form of preconditions and effects of events, and we show that it outperforms state-of-the-art supervised learning systems on the task of reconstructing RoboCup soccer games from their commentaries.
Introduction
Understanding a narrative involves more than just translating individual sentences; it requires building a coherent discourse model of the world. In a soccer game, for example, a commentary like "Pink goalie kicks off to Pink2" implies a change in ball possession that must be reflected in the model. Mapping these natural language descriptions to formal event sequences (e.g., pass, kick, or steal) is a significant challenge in AI, particularly because traditional supervised methods require expensive, manually annotated datasets.
This research addresses the "cold start" problem: learning to interpret narratives using only the text itself and a basic understanding of the domain's physical rules. The authors introduce ITEM (Iterative Event Mapping), a framework that leverages domain knowledge to guide an unsupervised learning process.
Methods
The ITEM algorithm is built upon two primary pillars: logical domain knowledge and iterative self-training.
Domain Knowledge as a Logical Bias
Instead of human-provided labels, the system uses a small set of STRIPS-like axioms that define the preconditions and effects of 16 soccer event types. For instance, a player cannot kick the ball unless they are currently holding it. This requirement for "physical coherence" acts as a filter, allowing the system to discard interpretations that are logically impossible within the game state.
Iterative Learning (IterTrain)
ITEM employs a "self-training" approach similar to Hard Expectation-Maximization (EM). The process begins by making initial guesses for labels based on simple heuristics, such as the edit distance between words and event names, while ensuring physical possibility. A binary logistic regression classifier is then trained on these guesses. The system iteratively re-scores and updates the labels using the classifier's output, repeating the loop until the model parameters converge.
Inference via Dynamic Programming
During the test phase, the system uses a Viterbi-like algorithm to determine the event sequence that maximizes the classifier's utility score while strictly adhering to physical constraints. This ensures that the final output is not only linguistically plausible but also physically consistent.
Results and Evaluation
The system was evaluated using human-written English commentaries from four RoboCup simulation league championship games (2001–2004), totaling 1,872 comments. The world model included 24 player constants, 10 state predicates, and 16 event types.
ITEM achieved a micro-averaged accuracy of 77.9% across the four games. Remarkably, this unsupervised approach outperformed the previous state-of-the-art supervised system, WGIM, which achieved only 70.3% despite having access to annotated training data.
As shown in the results table, even when the authors' physical consistency inference logic was applied to the supervised WGIM model (WGIM-Inference), it only reached 73.4%. This suggests that ITEM’s iterative learning process is fundamentally superior at capturing the relationship between natural language and world states.
Conclusion and Future Work
The core insight of this work is that the "logic of the world" can effectively replace human annotators in narrative understanding tasks. By enforcing temporal and physical consistency, the ITEM framework provides a robust method for learning from unlabeled text.
Future research will focus on moving beyond linear sequences to handle complex rhetorical structures like flashbacks, incorporating deeper syntactic parsing, and scaling the domain knowledge using massive semantic lexicons like VerbNet.
Summary updated