Structuring Human-AI Productive Interdependence by Strategic Level of Automation Selection for Qualitative Inquiry

Feng Zhou, Jacqueline Meijer-Irons, Ambar MurilloarXiv 2026 · DOI · arXiv

TL;DR

This paper argues that effective Human-AI collaboration in qualitative analysis isn't about maximizing automation, but about strategically structuring interdependence. They propose a framework for selecting the right "Level of Automation" (LoA) for different tasks, balancing interpretive risk and validation costs, to foster trust and preserve the human researcher's crucial role in meaning-making.

What Problem Does This Paper Address?

Qualitative analysis, a deeply interpretive and iterative process, faces a "crisis of scale" as the volume of data from sources like surveys and interviews overwhelms traditional manual methods. While Large Language Models (LLMs) offer a tempting solution for automation, a purely automated approach risks de-contextualizing data, flattening rich meaning, and sidelining the researcher's essential interpretive lens. The core challenge is how to leverage AI's scaling capabilities without sacrificing the nuance and rigor inherent in qualitative inquiry, leading to a new question: not "how much can we automate?" but "how should we structure the interdependence between AI and the human researcher?"

Methods

The paper introduces a formal framework for structuring Human-AI productive interdependence in qualitative inquiry, drawing on Interdependence Theory. This framework guides the selection of an appropriate Level of Automation (LoA) for different stages of the qualitative analysis process. The key to this selection is a calculated assessment of two factors:

  1. Risk of Interpretive Failure: The potential cost of an AI error to the final analysis. High-risk tasks (e.g., final thematic synthesis) require a low LoA, while repetitive, low-level tasks (e.g., open coding of a single comment) can tolerate a higher LoA.
  2. Validation Cost: The difficulty of validating the AI's work. Tasks requiring deep, tacit knowledge for evaluation demand a low LoA, whereas tasks easily checked against source text can accommodate a higher LoA.

The authors also present a case study demonstrating a three-phase workflow that implements this framework, emphasizing task-dependent agency and causal communications to maximize joint performance. They highlight the use of the Clue And Reasoning Prompting (CART) framework with Many-Shot In-Context Learning for transparent AI "thought processes."

Datasets

Not discussed in the paper. The case study describes using internal survey data from Google employees.

Key Insights and Ideas

  • Interdependence, Not Automation: The central thesis is that effective Human-AI collaboration in qualitative analysis is fundamentally an interdependence problem, not an automation problem.
  • Strategic LoA Selection: The framework for choosing the Level of Automation (LoA) based on risk and validation cost is a novel approach to designing Human-AI workflows for qualitative tasks.
  • Calibrated Trust as an Emergent Property: Trust in AI is not a design target but an emergent property of a well-structured, transparent, and methodologically sound interdependent system.
  • Human as Strategic Interpreter: The human researcher's role is crucial for the subjective transformation process—evaluating AI suggestions, infusing context, and weaving the final narrative. AI acts as a tactical analyst.
  • Transparency is Key: Mechanisms like the CART framework, which force the AI to "show its work," are vital for building shared cognition and enabling mutual validation.

Key Results

The paper presents a case study of a Human-AI workflow for analyzing large-scale employee survey data. The workflow successfully scaled data analysis while maintaining depth. Key outcomes include:

  • Phase 1 (Foundation): AI performed open coding on 900 comments after human-defined seed examples, followed by manual verification, establishing a "golden dataset" and a trustworthy foundation.
  • Phase 2 (Tactical Analyst): AI operated at a high LoA (Level 7-10) for high-volume open coding, leveraging CART for transparent reasoning.
  • Phase 3 (Strategic Interpreter): AI produced "AI Insights" (Level 4 LoA) of thematic clusters, with the human researcher performing the core strategic role of interpretation and narrative construction.

The authors propose three design principles derived from their experience: Mandate Bi-Directional Validation, Use Orienting Guidance (LoA 3-5), and Design for Asymmetry to Isolate the Transformation Process.

Limitations and Future Work

  • Limitations:
    • The AI-to-Human validation loop in their case study was passive, meaning the AI didn't explicitly flag when it struggled, which could have indicated issues with the research instrument.
  • The separation of tactical (AI) and strategic (human) roles was implicit, potentially leading naive users to overestimate AI's understanding.
  • The framework proposes static LoA for each task, which could be more dynamic.
  • Future Work:
    • Developing systems that can infer researcher trust and dynamically adjust the interdependent structure.
  • Investigating effective system-initiated strategies for trust repair when AI violates trust (e.g., through hallucinations).
  • Exploring how AI can proactively suggest methodological refinements or new codes when confidence is low.

Summary updated

Abstract

While Large Language Models (LLMs) offer a solution to the scale-versus-depth dilemma in qualitative analysis, the paradigm of maximizing automation is fundamentally at odds with the interpretive nature of qualitative inquiry. We argue that effective Human-AI collaboration is not an automation problem, but an interdependence problem. This paper reframes the design of "co-data" systems through the lens of Interdependence Theory, proposing a formal framework to structure human-AI productive interdependence. The framework guides the selection of an appropriate Level of Automation (LoA) for different stages of the qualitative analysis process by assessing task risk and the cost of validation. We present a case study where this framework led to a deliberately interdependent workflow, fostering the calibrated trust necessary for rigorous analysis. We conclude by presenting three design principles that instantiate this framework, demonstrating how to leverage AI as a powerful partner while preserving the human researcher's irreplaceable role in the transformation process of meaning-making.
BibTeX
@misc{zhou2026structuring,
  author = {Feng Zhou and Jacqueline Meijer-Irons and Ambar Murillo},
  title = {{Structuring Human-AI Productive Interdependence by Strategic Level of Automation Selection for Qualitative Inquiry}},
  year = {2026},
  eprint = {2605.27634},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.27634},
}
Figure