Abstract
Effective communication often relies on elided commonsense knowledge—details that experts assume but novices require. This paper introduces KIDSCOOK, a dataset that pairs abstract cooking instructions with detailed, "child-directed" steps. By benchmarking various models, including large language models and neural transduction architectures, the authors explore how AI can bridge the gap between high-level goals and atomic procedural actions.
Introduction: The Challenge of Elided Knowledge
When a recipe says "drain the pasta," an adult knows to find a strainer, place it in the sink, and carefully pour the pot's contents. This implicit knowledge is often missing from standard NLP corpora, which tend to be "monolithic" in their level of detail. For AI to function in real-world environments, it must master hierarchical script knowledge—the ability to decompose abstract goals into concrete, physically grounded actions.
The KIDSCOOK Dataset
To address this data gap, the researchers developed KIDSCOOK. Built upon the YouCookII dataset, KIDSCOOK uses a "child-directed" heuristic: annotators were asked to explain video captions to a child. This constraint successfully elicited the "how-to" knowledge—tools, temporal ordering, and basic physics—that adults typically skip.
KIDSCOOK is significantly richer than its predecessor, with captions roughly four times longer and containing over 1,500 new concepts. This expansion highlights the vast amount of procedural information usually hidden in standard instructional text.
Methods and Model Architectures
The task was framed as a sequence-to-sequence mapping problem (Abstract $\rightarrow$ Concrete). The study compared:
- Standard Seq2Seq: Baselines with and without attention.
- Neural Transduction Models: These models utilize latent hard alignment variables to learn specific phrase-to-phrase correspondences, prioritizing interpretability.
- Fine-tuned GPT: A state-of-the-art generative model used for comparison.
Evaluation was conducted via a Cloze Task, where models filled in missing content words (tools and procedures) in concrete instructions based on abstract prompts.
Results: Accuracy vs. Interpretability
While the fine-tuned GPT model achieved the highest accuracy (38.01% exact match), it functioned largely as a "black box." In contrast, the neural transduction models provided "Viterbi alignments"—clear, human-readable maps showing which part of an abstract goal triggered specific concrete actions.
Despite GPT's performance, it occasionally produced nonsensical outputs, such as suggesting a "knife" to add sauce. The transduction models, while less accurate, offered superior alignment quality, with concise and logical spans.
Conclusion and Future Directions
The KIDSCOOK benchmark reveals that while modern LLMs are proficient at pattern matching, they still struggle with the functional commonsense required for grounded reasoning. Future work aims to link these hierarchical scripts to visual features or physical simulators, moving toward agents that truly understand the "why" and "how" of human procedures.
Summary updated