Agentic Path Maps: Exposing an Agent's Traversal Through the Garden of Forking Paths

Madeleine Grunde-McLaughlin, Hafsah Usman, Daniel Mitroshkov, Emily Reif, Nino Migineishvili, Lakshya Chauhan. Kevin Yu, Jeffrey Heer, Daniel S. WeldarXiv 2026 · arXiv

TL;DR

AI agents often make silent, arbitrary assumptions to complete underspecified tasks, leading to results that are technically correct but potentially useless to the user. This paper introduces Agentic Path Maps, a framework for visualizing the "untaken roads" an agent considered, demonstrating that exposing these alternatives significantly reduces user overreliance and encourages more critical engagement with AI outputs.

What Problem Does This Paper Address?

When you ask an AI agent to "find the best price for aquarium tickets," it has to make a dozen micro-decisions: Are you a state resident? Are you going on a weekend? How many kids are with you? If the task is underspecified, the agent fills in the gaps with assumptions.

The authors describe this as the "Garden of Forking Paths." While an agent might return a "task-valid" output (a real price found on a website), it might not be "user-valid" (the price for your specific situation). Currently, we face a binary choice: either the agent executes silently (risking incorrect assumptions) or it interrupts the user constantly for clarification (creating "interruption fatigue").

This paper explores a third way: post-hoc transparency. How can agents communicate the impactful decisions they made—and the alternatives they ignored—after the task is done?

Figure 1: Agentic workflows make assumptions about user’s goals to complete underspecified tasks. Although the agent’s found output may be valid under the task constraints, it may not be valid given the users’ context and preferences. In this paper, we contribute Agentic Path Maps to communicate untaken alternative paths to users. We first distill a space of designs for generating and studying Agentic Path Maps. We then manipulate one dimension, the quantity of paths shown, in an empirical user study. We find that participants overrelied less and were more cognitively engaged when shown alternative paths.

Methods

The researchers approached this in two stages:

  1. Design Space Distillation: They reviewed over 2,400 papers across domains like multiverse analysis, counterfactual explanations, and decision trees to identify eight key dimensions for communicating alternatives (e.g., Objectives, Structure, Timing, Quantity).
  2. Empirical User Study: They conducted a study with 48 participants in the domain of data analysis. Using a data agent (DataVoyager), they manipulated the quantity of paths shown to the user:
  3. Baseline: A single "standard" output (often an extreme value).
  4. Treatment: An Agentic Path Map showing either 3 or 9 alternative analysis pathways.

To measure impact, they used Weight of Advice (WoA)—a metric quantifying how much a user shifts their prior belief toward the AI's suggestion—and a validated Cognitive Load questionnaire.

Datasets

The study utilized the BLADE benchmark, which provides expert-validated decision spaces for data-driven science. Specifically, they used two datasets:

  • Affairs: Investigating if having children affects the rate of extramarital affairs.
  • Crofoot: Measuring how the size and location of capuchin monkey fights affect the likelihood of winning.

These datasets were chosen because they allow for "multiverse analysis," where different defensible statistical choices (like which variables to control for) can lead to wildly different results.

Key Insights and Ideas

  • The Anchoring Trap: Without seeing alternatives, users almost never consider other analysis choices on their own. They anchor heavily to the first "task-valid" result the AI provides.
  • Productive Effort (Germane Load): While showing more paths increases the mental effort required (Intrinsic and Extraneous Cognitive Load), it also increases Germane Cognitive Load. This is "good" mental effort—the kind associated with actual learning and critical reflection.
  • The "Three-Path" Sweet Spot: Increasing the number of shown alternatives from three to nine showed diminishing returns. A sample of three paths (e.g., minimum, median, and maximum outcomes) provided a strong middle ground between transparency and information overload.
  • Agency through Alternatives: Participants reported feeling more agency when seeing Path Maps. Instead of just "relying on the AI," they felt they could "critically think about why a method was chosen."

Key Results

The study found strong statistical support for the benefits of Path Maps:

  • Reduced Overreliance (H1): There was a 99.92% probability that participants placed a lower Weight of Advice on extreme outputs when shown alternatives (Effect size: -0.63, very large).
  • Better Sensemaking (H2): Participants' final beliefs ended up significantly closer to the "multiverse median" (the consensus of all possible paths) rather than being pulled toward a single extreme AI output (96.45% probability).
  • Increased Engagement (H4): Users spent significantly more time reviewing the interface when shown alternatives (100% probability), indicating they were actually auditing the agent's work rather than rubber-stamping it.
Participants' posterior beliefs (orange) shifted closer to the median and away from extremes when shown multiple paths.

Limitations and Future Work

  • Computational Cost: Generating 9 or 100 paths is significantly more expensive than generating one. Future work needs to address how to select the most "impactful" paths to generate.
  • Agent Reliability: The study used expert-validated paths. In the real world, an AI agent might generate invalid paths, which could further confuse users.
  • Domain Specificity: The results are grounded in data analysis. The authors note that for irreversible tasks (like making a purchase), the cost-benefit analysis of showing alternatives might change.
  • P-Hacking Risks: Providing too many alternatives might inadvertently encourage users to "cherry-pick" the result that matches their existing bias (p-hacking).

Summary updated

Abstract

When an AI agent executes an underspecified task, it embeds choices that may contradict unarticulated user context and preferences. However, interrupting the user for clarification on all such decisions is untenable. How might agents communicate impactful, but unexplored, paths to the user after execution? To investigate, we first distill a design space for communicating alternative paths, supporting the generation of new systems and structuring dimensions of future study. We then measure the impact of one vital design space dimension on user comprehension: the number of alternatives displayed. Through a user study (n=48) in the data analysis domain, we find that participants rarely consider alternative paths of their own volition, but AI overreliance reduces when alternative paths are visualized for them. By revealing paths the agent did not take, we demonstrate how users can better interpret results and incorporate personal context.
BibTeX
@misc{grundemclaughlin2026agentic,
  author = {Madeleine Grunde-McLaughlin and Hafsah Usman and Daniel Mitroshkov and Emily Reif and Nino Migineishvili and Lakshya Chauhan. Kevin Yu and Jeffrey Heer and Daniel S. Weld},
  title = {{Agentic Path Maps: Exposing an Agent's Traversal Through the Garden of Forking Paths}},
  year = {2026},
  eprint = {2611.04183},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04183},
}
Figure