Human--AI Interactions as First-Class Data: A Formal Interaction Model and Runtime for Explainable AI Collaboration

Michela Taufer, Kin Ng, Ian Lumsden, Aashish Pandey, Rajasi Gore, Angelina Ju, Sukanya Moudgalya, Ewa Deelman, Sai SwaminathanarXiv 2026 · arXiv

Treating an AI assistant like a simple chat interface is a bit like trying to understand a complex scientific experiment by only reading the lab notes' margins. You see the questions and answers, but you miss the actual experiment—the code that failed, the data that was being processed, and whether the scientist actually changed their approach based on the AI’s advice.

In their paper, "Human–AI Interactions as First-Class Data," Michela Taufer and her colleagues argue that we need to stop treating these interactions as transient "transcripts" and start treating them as durable, structured data products.

TL;DR

The authors introduce HAIDL, a formal language and state-transition model that captures the full context of human-AI collaboration—from the code error that prompted a request to the user's subsequent actions. By implementing this in a JupyterLab middleware called HAIDL-X, they demonstrate that treating interactions as "first-class data" enables deep, reproducible analysis of how AI actually impacts complex work like data engineering.

What Problem Does This Paper Address?

Currently, when you use an AI assistant in a coding environment, the logs usually only record the prompt and the response. This creates a massive "context gap."

For example, if two students both have zero interactions with an AI, one might be a "self-sufficient" expert, while the other might be a "disengaged struggler" who is stuck but doesn't know how to ask for help. Standard logs can't tell them apart. Similarly, we often can't tell if an AI's suggestion actually helped a user fix a bug or if the user just ignored it and moved on. This lack of structured data hinders reproducibility, explainability, and our ability to build better, more adaptive AI tools.

Methods

The researchers developed two primary components:

  1. HAIDL (Human-AI Interaction Description Language): A formal grammar (inspired by Backus-Naur Form) that defines an interaction as a single, structured object. Unlike a simple chat log, a HAIDL object captures seven dimensions: record metadata, the user request, the prompt payload, the AI response, user feedback, the problem context, and model metadata.
  2. HAIDL-X Middleware: A runtime implementation for JupyterLab. It acts as a layer between the user and the AI, automatically capturing the notebook's state (code, errors, outputs) whenever a request is made and persisting the entire exchange into a queryable repository.

The core of their approach is modeling interaction as a state transition: Lifecycle of a human–AI interaction As shown in Figure 1, the interaction begins with a computational state ($S_i$), involves the request/response/assessment cycle, and results in a new state ($S_{i+1}$).

Fig. 1: Lifecycle of a human–AI interaction. A user request arises from a computational state, the AI returns guidance, and the user evaluates or acts on it. HAIDL represents the full lifecycle as a structured artifact whose outcome seeds the next interaction.

Datasets

The authors evaluated their system through a semester-long deployment in a university Data Engineering course.

  • Participants: 35 students.
  • Tasks: Four major programming assignments ranging from algorithmic problem solving to distributed data processing with PySpark.
  • Data Volume: Approximately 800 recorded interactions and 140 student-assignment pairs.
  • Alignment: They combined the HAIDL interaction logs with independent notebook execution traces (which cells were run, which failed, etc.) to create a complete picture of student behavior.

Key Insights and Ideas

  • Interaction Counts are Deceptive: The researchers found that the raw number of prompts has almost no correlation with success. A student might prompt the AI 20 times because they are exploring deeply, or because they are hopelessly lost.
  • Deriving "Knowledge" from Data: By using formal derivation rules, the authors could turn raw logs into "Behavioral States." For instance, they could identify a "Scaffolded Learner" by looking for low performance paired with productive help-seeking (error recovery and next-step requests).
  • Resolving Ambiguity: The most powerful insight was the ability to distinguish between different types of "non-users." By combining execution data with interaction data, they could separate students who were progressing independently from those who were struggling in silence.

Key Results

  • Negligible Overhead: The HAIDL-X middleware adds very little latency. The "capture overhead" (the time taken to package the state and save the data) remained under 25ms across various AI providers and concurrency levels.
  • Efficient Storage: Interaction objects are lightweight, with a median size of just 4.53 KB, making long-term persistence practical for large-scale deployments.
  • Qualitative Mapping: The researchers successfully mapped student behavior into four distinct learning profiles (HPHE, HPLE, LPHE, LPLE) based on engagement and performance, as detailed in their derived interaction events table.
TABLE I: Derived interaction events and notebook-derived learning profiles by assignment.

Limitations and Future Work

The authors are careful to note that this paper does not claim that AI improves learning; rather, it provides the tools to measure that impact accurately.

Limitations: * The current study is limited to a classroom setting and computational notebooks. * The "assessment" phase often relies on implicit signals (like whether a user ran a cell after a response), which can be noisy.

Future Work: * Expanding HAIDL to support multi-agent scientific workflows. * Using the captured data to build "adaptive AI" that can recognize when a user is in a "disengaged struggler" state and proactively offer different types of support. * Applying this model to auditing AI-assisted scientific discoveries to ensure every step of the collaboration is traceable and reproducible.

Summary updated

Abstract

AI assistants are increasingly embedded in computational notebooks, scientific workflows, software engineering tools, and learning systems, yet human-AI interactions are often treated as transient prompt-response exchanges rather than durable data products, hindering reproducibility, explainability, replay, behavioral analysis, and adaptive AI support. We argue that these interactions form a new data class requiring formal representation and introduce the Human-AI Interaction Description Language (HAIDL), a formal interaction model with a Backus-Naur Form-inspired concrete syntax. HAIDL encodes each interaction as a structured object capturing computational context, user request, prompt payload, AI response, feedback, problem context, provenance, and model metadata. We integrate this vision into HAIDL-X, a middleware for JupyterLab that instantiates HAIDL by capturing user-bot state, building contextualized prompts, interfacing with AI providers, collecting feedback, and storing interactions in a repository. We add a learning layer over HAIDL that maps interaction objects and notebook execution traces into reproducible measures of user behavior, including engagement, performance, help-seeking, error-oriented use, and learning personas. HAIDL-X provides a foundation for treating human-AI collaboration as structured, queryable, replayable, analyzable data.
BibTeX
@misc{taufer2026human,
  author = {Michela Taufer and Kin Ng and Ian Lumsden and Aashish Pandey and Rajasi Gore and Angelina Ju and Sukanya Moudgalya and Ewa Deelman and Sai Swaminathan},
  title = {{Human--AI Interactions as First-Class Data: A Formal Interaction Model and Runtime for Explainable AI Collaboration}},
  year = {2026},
  eprint = {2610.40006},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.40006},
}
Figure