"The Diagram is like Guardrails": Structuring GenAI-assisted Hypotheses Exploration with an Interactive Shared Representation

Zijian Ding, Michelle Brachman, Joel Chan, Werner Geyer

Introduction

While Generative AI (GenAI) has proven adept at execution-oriented tasks like code generation or chart creation, it often falters during the "conceptual" phase of data analysis. This phase involves the messy, non-linear process of determining which questions are worth asking and which hypotheses are worth testing. Traditional chat-based interfaces often trap analysts in linear workflows, leading to cognitive fixation and difficulty tracking multiple exploratory paths. This paper introduces a novel interactive system that replaces the linear chat log with a structured, shared node-link diagram, acting as "guardrails" for collaborative human-AI hypothesis exploration.

The Challenge of Linear Interfaces

Current AI assistants typically rely on scrolling chat windows. For data analysts, this creates two major hurdles:

  1. Fixation: Users often get stuck on a single line of inquiry suggested by the AI, missing broader patterns or alternative explanations.
  2. Cognitive Load: It is difficult to maintain a mental map of multiple "what-if" scenarios or backtrack to previous ideas within a long, linear transcript.

To address this, the researchers designed a "shared representation"—a workspace both the human and the AI can manipulate—that supports non-linear exploration, allowing analysts to explore the breadth of a dataset without losing depth.

System Architecture

The prototype system consists of three integrated components designed to bridge the gap between high-level intent and concrete data exploration:

  • Intent-Hypothesis Interpreter: Powered by GPT-4o, this component translates high-level user goals (e.g., "analyze income inequality") into specific, testable hypotheses.
  • Ordered Node-link Diagram: Instead of a chat log, the system displays hypotheses as a tree. Vertical levels represent iterations (depth), while horizontal branches represent alternative paths (breadth).
  • Information Hints Explorer: To help users decide which branch to follow, the system provides "hints"—preliminary data visualizations generated via the LIDA framework and supporting text from literature via Retrieval-Augmented Generation (RAG).

Methodology

The researchers evaluated the system through a qualitative design probe involving 22 professional data analysts and researchers. Participants utilized the tool to explore the 2020 US Census dataset, which contains approximately 40,000 rows and 15 columns (including age, education, marital status, and income). This dataset was selected for its ability to support complex, multi-layered hypotheses regarding socioeconomic factors.

Key Insights: The "Guardrails" Effect

The study revealed several critical ways in which the structured representation improved the analytical process:

Mental Scaffolding and Bias Mitigation

Participants found that the tree structure acted as a mental scaffold. It limited their "imagination" just enough to keep them focused on data-driven paths while providing the freedom to jump between different branches of thought. By presenting five initial hypotheses at once, the system helped analysts avoid "prior bias." Seeing multiple options simultaneously prompted users to consider variables—such as marital status—that they might have otherwise ignored in favor of more obvious factors like education level.

Visual Grounding

One of the most critical findings was that AI-generated visualizations were "affirming." Analysts felt that seeing a preliminary chart—even a simple one—transformed an abstract AI suggestion into a credible, data-backed concept. This visual grounding made users significantly more willing to pursue a suggested hypothesis.

Embracing Non-Linearity

Data exploration is naturally "loopy" rather than linear. The ability to "backtrack"—returning to a higher-level node to start a completely new branch of inquiry—was a primary behavior observed during the study. The node-link diagram facilitated this by making the entire history of the exploration visible and interactive.

Results and User Sentiment

The system demonstrated high engagement and structural complexity during the sessions:

  • Engagement: Participants generated an average of 21.82 hypotheses per session.
  • Structure: The resulting diagrams reached an average maximum depth of 3.86 levels and a maximum breadth of 8.09 branches.
  • Efficiency: Users noted that the integration of hypotheses and charts reduced "context switching," as they did not have to leave the brainstorming interface to verify if a hypothesis was supported by the data.
  • Adoption: 20 out of 22 participants reported they would use such a tool in their future work.

Limitations and Future Work

Despite the positive reception, the researchers identified several areas for improvement. As the number of hypotheses grows, the node-link diagram risks becoming a "hairball" that is difficult to navigate. Future iterations may explore dynamic rescaling or "folding" branches to manage this complexity. Additionally, participants expressed a desire for more direct control, such as the ability to manually select specific data columns for the AI to focus on.

While this study was primarily qualitative, it lays the groundwork for future quantitative A/B testing to compare the quality and novelty of hypotheses generated with structured diagrams versus traditional chatbot interfaces.

Summary updated

BibTeX
@inproceedings{ding2025diagram,
  author = {Zijian Ding and Michelle Brachman and Joel Chan and Werner Geyer},
  title = {{"The Diagram is like Guardrails": Structuring GenAI-assisted Hypotheses Exploration with an Interactive Shared Representation}},
  booktitle = {Creativity \& Cognition},
  year = {2025},
  doi = {10.1145/3698061.3726935},
}
Figure