Abstract
Stochastic planning using Factored Markov Decision Processes (MDPs) offers a robust framework for decision-making under uncertainty but suffers from high barriers to entry for non-technical domain experts. This paper presents an integrated suite of tools—including the HELL elicitation tool and the PlanIt visualizer—that translates qualitative human intuition into quantitative models. By utilizing a "bowtie" model for Bayesian network construction and archetypal templates for preference elicitation, the system enables users to build and navigate complex stochastic plans without deep mathematical expertise.
Introduction
Planning in uncertain environments is often modeled using Markov Decision Processes (MDPs). While mathematically robust, "factored" MDPs—which decompose the state space into multiple variables—are notoriously difficult to construct and interpret. Two primary hurdles exist: the knowledge gap, where building models requires an exhausting number of conditional probabilities, and the interpretability gap, where generated policies appear as "black boxes" to non-technical users. This research introduces a pipeline that moves from qualitative human intuition to quantitative machine planning, and back to a visual human-centric display.
System Architecture
The proposed suite consists of interconnected tools designed to handle the full lifecycle of MDP development, from initial domain elicitation to final plan visualization.
The architecture bridges the gap between the user's conceptual model and the underlying mathematical engine. Elicitation tools like HELL, POET, and DIET feed into a central planner, which then outputs results to PlanIt for user interaction.
Qualitative Elicitation: The Bowtie Model
A core innovation is the High Level Elicitor (HELL) tool, which utilizes a "bowtie" model to build Bayesian network fragments. Rather than requesting raw probability values, HELL asks users to identify "predictors" of an action's success and the "attributes" likely to change upon success.
As shown in the figure, the bowtie structure (input nodes → action → output nodes) allows users to specify qualitative influences (e.g., "being tired" increases the likelihood of "taking a nap"). These inputs are automatically converted into quantitative dynamic Bayesian networks, significantly reducing the cognitive load on domain experts.
Preference Elicitation and Plan Visualization
To further streamline the process, the system employs preference elicitation tools (DIET and POET) that use "archetypes"—pre-defined templates such as a "Social" vs. "Technical" conference attendee. Users can select and modify these templates instead of defining utility functions from scratch.
Once a plan is generated, the PlanIt visualizer allows users to "walk through" the policy. It represents states and outcomes using a geometric metaphor: the size of a box represents its probability, while the color indicates its utility. This provides a pre-attentive way for humans to process complex branching paths in a stochastic environment.
Evaluation: Conference Attendance Model
The suite was evaluated using a conference attendance scenario (based on AAAI/IAAI). The model included variables such as session topics, topic saturation, and time. Actions included attending specific talks or taking breaks, with the goal of maximizing utility based on coverage while managing constraints like "tiredness." The integration proved successful, allowing a complex scheduling problem to be modeled and navigated without requiring the user to write code or manually populate probability tables.
Conclusion and Future Work
This work represents a significant step toward making stochastic planning accessible. By bridging the qualitative-quantitative divide, the system empowers domain experts to leverage MDPs effectively. Future research will focus on scaling the bowtie model to higher-dimensional state spaces and conducting formal user studies to quantify the efficiency gains provided by this integrated approach.
Summary updated