Introduction
Inferring a human's high-level goal from a sequence of discrete actions—such as recognizing that "boiling water" and "adding pasta" constitutes "cooking pasta"—is a fundamental challenge in artificial intelligence known as Multi-axis Event Process Typing. This task requires the AI to predict both the overarching Action (e.g., COOK) and the primary Object (e.g., PASTA) of a process.
Historically, state-of-the-art methods like P2GT treated this as a "learning-to-rank" problem, relying on external dictionaries like WordNet and complex Word Sense Disambiguation (WSD) systems. These approaches were often brittle, constrained by fixed vocabularies, and required heavily filtered datasets that stripped away vital context. This paper introduces STEPS (Semantic Typing of Event Processes with a Sequence-to-sequence approach), which reformulates the task as a direct text generation problem.
Methodology
STEPS moves away from ranking systems and fixed dictionaries in favor of a generative architecture based on BART (a denoising autoencoder for pre-training sequence-to-sequence models).
- Architecture: The authors evaluated both BART-Base (STEPS<sub>BB</sub>) and BART-Large (STEPS<sub>BL</sub>).
- Input/Output: The model receives a sequence of events wrapped in special delimiters (e.g.,
<e1> boil water </e1>) and is trained to generate a target string containing the action and object tags:how to <a> cook </a> <o> pasta </o>. - End-to-End Training: Unlike previous models, STEPS does not require external glosses or definitions. It leverages the internal linguistic knowledge BART acquired during pre-training to map event sequences directly to natural language labels.
Datasets
The study utilizes data from wikiHow, comparing two versions:
- Preprocessed Dataset: A heavily filtered version (62,277 instances) averaging only 2 steps per process.
- Raw Dataset: A larger, noisier version (108,027 instances) containing full procedural flows averaging 5 steps per process.
The dataset includes a wide distribution of labels, with 1,336 action types and 10,441 object types, many of which are "few-shot" (appearing fewer than 10 times).
Key Results
STEPS outperformed previous state-of-the-art (SOTA) models by a staggering margin. On the raw dataset, STEPS-Large achieved a Mean Reciprocal Rank (MRR) of 51.55 for Action Typing (nearly double the previous best of 29.11) and an MRR of 35.84 for Object Typing (up from 15.70).
Notably, STEPS demonstrated exceptional efficiency in low-data scenarios. As shown in the figure above, even when trained on only 10% of the data, STEPS-Large outperformed the previous SOTA trained on the full dataset.
Key Insights and Discussion
More Context is Better
A striking finding was that STEPS performed significantly better on raw, unfiltered data than on "cleaned" versions. While previous researchers believed filtering was necessary to reduce noise, STEPS proved that the extra context in longer event sequences helps the model triangulate intent more effectively.
Order Invariance
The authors discovered that the specific chronological sequence of events is less critical than their presence. Shuffling or inverting the order of events did not significantly degrade performance, suggesting that for intent inference, the collection of actions is the primary signal.
Action vs. Object Sensitivity
Action typing was found to be more sensitive to the volume of training data than object typing. This is likely due to the highly skewed distribution of actions in the wikiHow dataset, where a few common verbs dominate the training signal.
Conclusion
By treating event process typing as a sequence-to-sequence generation task, STEPS achieves a massive performance leap over traditional ranking methods. The model's ability to leverage pre-trained linguistic knowledge allows it to handle free-form descriptions and rare labels with unprecedented accuracy. Future work will explore extracting these intents from unstructured text and applying these inferred goals to improve dialogue systems and story completion tasks.
Summary updated