Investigating Interaction Modes and User Agency in Human-LLM Collaboration for Domain-Specific Data Analysis

Jiajing Guo, Vikram Mohanty, Jorge Henrique Piazentin Ono, Hongtao Hao, Liang Gou, Liu Ren

Introduction

While Large Language Models (LLMs) like GPT-4 have demonstrated remarkable proficiency in general-purpose coding, their utility in domain-specific data analysis—such as manufacturing, finance, or specialized sensor telemetry—is often hindered by unique terminologies and complex data structures. This research investigates the design space of AI-powered data analysis tools, specifically focusing on how the level of user agency and the interaction mode (structured vs. open-ended) influence the performance and trust of professional data scientists.

Methodology: Comparing OHA and SLA

The researchers developed two distinct "design probes" to represent opposite ends of the interaction spectrum:

  1. Open-ended High Agency (OHA): A traditional chat-based interface where the user maintains total control. The user is responsible for writing prompts, providing context, and manually executing generated code within their own environment.
  2. Structured Low Agency (SLA): A guided system that leads the user through a systematic six-step pipeline: data description, domain knowledge injection, prompt enhancement, plan selection, automatic execution/debugging, and insight review.

To evaluate these modes, the study engaged nine professional data scientists from Bosch. Participants performed analysis tasks on real-world, "messy" internal enterprise data from manufacturing and sensor domains. This setup allowed the researchers to observe how experts react to LLM outputs when dealing with specialized context and potential model errors.

The Agency Paradox

A central finding of the study is what the authors term the "Agency Paradox." Participants were highly receptive to delegating "execution-level" tasks—such as writing boilerplate code, resolving syntax errors, and generating plots—to the LLM. However, they remained protective of their "planning agency." While they appreciated the AI suggesting analytical strategies, they insisted that the final decision regarding the analytical path must remain with the human expert.

Key Insights

The Burden of Context

In the open-ended (OHA) mode, experts found it "cumbersome" to manually provide the LLM with necessary metadata and sample rows. Conversely, the structured (SLA) mode was praised for its ability to "fill the gaps" by forcing a systematic approach to context injection, which improved the relevance of the LLM's output.

Fragile Trust and the Need for Transparency

Trust emerged as a significant barrier, particularly in the automated SLA mode. When the system moved autonomously from a question to a final result, users often felt they could not trust the output without re-performing the work themselves. This "black box" effect suggests that automation can sometimes defeat its own purpose if it lacks transparency.

Behavioral Shifts

The study observed that in the structured mode, users tended to be less interactive and more likely to accept the system's first suggestion due to time constraints, even when they harbored doubts about the underlying logic. This highlights a risk of "automation bias" in structured workflows.

Conclusion and Future Directions

The research concludes that professional data scientists do not want full automation; rather, they require a "Verification Sandbox." This would include:

  • Reasoning Transparency: Visualizing the LLM's internal "reasoning cycles" (such as the ReAct framework).
  • Manual Intervention: The ability to tweak code or data mid-process within a guided workflow.
  • Hybrid Systems: Tools that allow users to toggle between structured guidance for efficiency and open-ended exploration for deep-dive analysis.

Future work will focus on expanding these findings to domain experts with lower coding proficiency and developing visual tools, such as flowcharts, to help users validate AI-generated logic more effectively.

Summary updated

BibTeX
@inproceedings{guo2024investigating,
  author = {Jiajing Guo and Vikram Mohanty and Jorge Henrique Piazentin Ono and Hongtao Hao and Liang Gou and Liu Ren},
  title = {{Investigating Interaction Modes and User Agency in Human-LLM Collaboration for Domain-Specific Data Analysis}},
  booktitle = {CHI Extended Abstracts},
  year = {2024},
  doi = {10.1145/3613905.3651042},
}
Figure