TL;DR
This paper introduces an AI assistant designed to guide novice users through DIY projects, like building a key rack, by providing adaptive, step-by-step instructions. It combines planning, ontological reasoning, and dialogue management to offer personalized guidance and answer user questions, making complex tasks more accessible and safer.
What Problem Does This Paper Address?
Many individuals are interested in DIY home improvement projects but are deterred by the complexity and potential dangers of using power tools like drills, saws, and sanders. Novice users often lack the knowledge to handle, configure, and safely operate these tools, leading to a perceived inability to undertake such tasks. This paper aims to bridge this gap by developing an intelligent assistant that provides comprehensive and adaptive instruction, empowering users to complete DIY projects successfully and fostering self-reliance.
Methods
The proposed assistant integrates three core capabilities:
- Planning: A planner generates a sequence of actions (a plan) for a given DIY project. This planner is designed to handle partially-ordered domains and can generate plans at different levels of abstraction, allowing for flexibility and adaptation to user experience. It also incorporates safety checks to prevent hazardous actions.
- Ontological Reasoning: A knowledge base, structured as an ontology, stores factual information about tools, materials, and their properties. This ontology is used to:
- Retrieve relevant instructional content (text, images, videos) for each action.
- Answer factual questions from the user about tools and materials (e.g., "How can I identify this drill?").
- Ensure consistency between the procedural knowledge used by the planner and the factual knowledge conveyed to the user.
- Dialogue Management: A dialogue manager facilitates communication between the user and the planning/ontology modules. It presents the plan as a sequence of task slides, incorporating information from the ontology. The system adopts a user-driven interaction strategy, allowing users to ask for more details or explanations.
Datasets
Not discussed in the paper.
Key Insights and Ideas
- Hierarchical Abstraction for Instruction: The system leverages hierarchical planning to present instructions at varying levels of detail. This allows experienced users to grasp the overall flow while novices can delve into specific steps, catering to a wider range of skill levels.
- Separation of Procedural and Factual Knowledge: By storing procedural knowledge in the planning model and factual knowledge in a separate ontology, the system achieves better coherence and reusability. This separation also allows for dynamic configuration checks (e.g., matching drill bit to material) to be treated as factual knowledge, simplifying state management.
- Ontology-Driven Explanations: The use of an ontology to answer user questions about tools and materials provides context-aware and accurate information, enhancing the learning experience beyond just following instructions.
- Adaptive Assistance: The planning-based approach allows the assistant to adapt to different situations, including variations in available tools and materials, and to dynamically repair plans in response to user mistakes.
Key Results
The paper describes a functional demonstrator system that successfully guides users through a sample project (building a wooden key rack). While quantitative results are not detailed, the system's architecture demonstrates the feasibility of integrating planning, reasoning, and dialogue for effective instruction in complex technical domains. The system is designed to handle projects with potentially dozens of individual steps and offers alternative ways to complete tasks.
Limitations and Future Work
The authors acknowledge that user tests with the current demonstrator are underway. Future work includes allowing users to request changes to the generated plan, which would be handled via Linear Temporal Logic goals. The system's ability to handle a wide range of DIY projects and its effectiveness across diverse user skill levels will likely be further explored in future iterations.
Summary updated