Abstract
Collaboration between humans and robots to accomplish different kinds of tasks has been recently studied as a planning problem and several techniques have been developed to define and generate shared plans where humans and robots collaborate to achieve a common goal. However, current methods require the knowledge of the human about the plan under execution and an agreement between users and robots about their roles before the execution of the plan. In this paper, we propose an extension to the Hierarchical Agent-based Task Planner (HATP) that enables humans and robots to negotiate some aspects of the collaboration online during the execution of the plan. The proposed method is based on the automatic generation of a conditional plan in which missing information is acquired at execution time by means of sensing actions. The proposed method has been fully implemented and tested on a real robot performing collaborative tasks in an office-like environment.
Introduction
In the realm of human-robot collaboration (HRC), traditional automated planning often operates under a "closed world assumption," where the robot is expected to have complete knowledge of the environment and human intentions before starting a task. However, in real-world service robotics, this is rarely the case. Most existing systems require rigid pre-agreements on roles or rely on disruptive "re-planning" when assumptions fail. Re-planning can lead to socially awkward behaviors, such as a robot stopping abruptly to recalculate its next move.
This research, presented at ICAPS 2017, addresses this friction by enabling "on-line negotiation." Instead of assuming human preferences, the robot treats unknown factors as variables to be resolved during execution. By planning for uncertainty from the outset, the robot can proactively engage in sensing actions—like asking a question—to resolve ambiguities without breaking the flow of interaction.
Proposed Methodology
The authors introduce an extension to the Hierarchical Agent-based Task Planner (HATP) that generates conditional plans. The core of this approach is a four-stage pipeline designed to transform high-level goals into flexible, executable plans.
- Extended Domain Formulation: The domain is augmented with "execution variables"—boolean flags representing information unknown at planning time (e.g., whether a human prefers to perform a specific task).
- Multi-Plan Production: HATP generates a separate plan for every possible combination of these variables. For $k$ unknown variables, $2^k$ plans are produced.
- Translation to Petri Net Plans (PNP): Each generated plan is converted into a Petri Net Plan, a formalism that excels at representing concurrent actions and complex control flows.
- Plan Merging: A specialized "Merge Algorithm" identifies divergence points across the PNPs and inserts "sensing actions" (e.g., speech-based queries) to determine the appropriate branch to follow at runtime.
Execution and Plan Representation
The final output is a unified Petri Net Plan that encapsulates all possible execution paths based on the outcomes of negotiation actions. This representation allows the robot to handle complex task logic while maintaining the robustness required for real-world execution.
As shown in the PNP, the robot can initiate a sequence (e.g., AskT1T2) and, based on the human's response, transition through different states to reach a fork where tasks are executed in parallel or sequence as negotiated.
Evaluation and Key Insights
The system was evaluated using custom scenarios on a physical robot in an office environment, such as object retrieval and collaborative navigation through doors and elevators.
Proactive Uncertainty Management
Rather than waiting for commands, the robot becomes proactive. Because uncertainty is a first-class citizen in the plan, the robot can move toward a person specifically to initiate a negotiation, treating the acquisition of information as a necessary task step.
Socially Acceptable Execution
By utilizing conditional plans, the robot avoids the "stop-and-go" behavior typical of re-planning systems. This leads to more fluid and predictable interactions, which is crucial for non-expert users.
Computational Feasibility
Despite the potential for state explosion ($2^k$ plans), the authors demonstrated that for practical HRC tasks, the system generates and merges plans within seconds. In a complex navigation experiment, the system successfully managed a plan with over 400 actions and 2,000 edges.
Conclusion and Future Work
This work provides a robust framework for online negotiation in HRC. While the current implementation relies on grammar-based speech tools and faces scalability challenges with a high number of variables, it sets a foundation for more flexible human-robot partnerships. Future directions include integrating Large Language Models (LLMs) for more natural dialogue and conducting formal user studies to quantify the social impact of this proactive planning approach.
Summary updated