Between Obligations and Evidence: A Participatory Study of Translating the EU AI Act into Technical Requirements

Helma Torkamaan, Masoud Mansoury, Yanan Xin, Oscar Oviedo Trespalacios, Roberta Savella, Andrej Zwitter, Maria Heuss, Huy Son Nguyen, Marie-Therese Sekwenz, Moyi Li, Roberta Tamponi, Megha KhoslaarXiv 2026 · arXiv

The European Union’s Artificial Intelligence Act (AI Act) is a landmark piece of legislation, shifting AI governance from voluntary "ethics-by-design" to mandatory "compliance-by-design." But for the engineers and researchers tasked with building these systems, a massive gap remains: How do you translate high-level legal concepts like "appropriate accuracy" or "meaningful human oversight" into lines of code and testable metrics?

A new paper, "Between Obligations and Evidence," explores this translation gap through a participatory study with 32 experts across tech, law, and policy. The findings suggest that compliance is not a static checklist but a complex, sociotechnical moving target.

TL;DR

The paper investigates the challenges of translating the EU AI Act's legal requirements into concrete technical specifications. Through a participatory workshop, the authors reveal that compliance evidence is highly fragile, often invalidated by organizational pressures, fragmented responsibilities, and the inevitable "drift" of real-world data.

What Problem Does This Paper Address?

The EU AI Act classifies certain AI applications (like those in healthcare, recruitment, or public transport) as "high-risk," subjecting them to strict requirements for data governance, transparency, and robustness. However, the Act is a "horizontal" framework—it provides general rules but doesn't specify exactly which metrics an engineer should use or what constitutes "sufficient" evidence of compliance.

The authors address the translation problem: the difficulty of moving from normative legal principles to measurable system properties. They argue that technical evidence alone (like a fairness score) is insufficient if it doesn't account for how the system is actually used in a messy, human-centric organizational workflow.

Methods

The researchers conducted a full-day participatory workshop involving 32 participants from academia and industry. The methodology was designed to move beyond abstract theory by using three realistic "high-risk" scenarios:

  1. Healthcare: A multimodal pain estimation system for patients.
  2. Recruitment: An AI assistant for hiring.
  3. Public Transport: A harassment and violence prevention assistant.

The workshop followed a structured seven-phase process (see Figure 1 below), including "adversarial" sessions where teams used fishbone (Ishikawa) diagrams to identify how seemingly compliant systems could fail in practice.

The workshop consisted of seven phases, moving from shared understanding to adversarial stress-testing of compliance.

Datasets

As a qualitative participatory study, this paper does not use traditional machine learning benchmarks. Instead, the "dataset" consists of:

  • Pre-workshop registration responses from 46 experts.
  • Completed worksheets from the workshop teams mapping AI lifecycle stages to regulatory provisions.
  • Fishbone diagrams and "worst-case scenario" narratives generated by participants.
  • Transcribed group discussions and panel notes.

Key Insights and Ideas

The study identified several "structural tensions" (Figure 6) and recurring themes where the translation process stalls:

  • The "Human-in-the-Loop" Fallacy: Simply having a human oversee an AI doesn't guarantee safety. Participants noted that under time pressure or with limited information, humans often default to "automation bias"—trusting the system because they lack the capacity to challenge it.
  • Metrics vs. Meaning: A system can have a "favorable" fairness metric while still being discriminatory in practice. For example, aggregate fairness scores in recruitment can conceal the unfair treatment of individual applicants.
  • Fragility of Evidence: Compliance evidence has an expiration date. A system validated on one patient population may become non-compliant if the hospital’s demographics shift or if the sensor hardware is updated.
  • Fragmented Responsibility: AI systems are built and operated by chains of actors (providers, vendors, deployers). Often, the person who sees the harm (the deployer) lacks the technical logs to understand it, while the person who built the model (the provider) lacks access to real-world outcomes.
The study identified five core structural tensions between innovation, regulation, metrics, and expertise.

Key Results

The authors synthesized their findings into three interrelated dimensions of the translation problem:

  1. Technical: Representativeness and bias cannot be assessed through data quality alone; they are dependent on the "construct" being measured and the context of use.
  2. Sociotechnical: Oversight is an organizational capability, not just a UI feature. It requires clear authority, sufficient time, and workable escalation procedures.
  3. Legal-Evidentiary: Technical documentation does not automatically equal legal compliance. Organizations must justify why specific assessments support specific legal claims.

One striking qualitative result was the call for "system sunsetting clauses": explicit agreements to discontinue a system if the conditions necessary to maintain its compliance (like the ability to retrain or monitor) can no longer be met.

Limitations and Future Work

The study is limited by its relatively small, Europe-centric sample size and its reliance on hypothetical (though realistic) scenarios.

Future work needs to focus on creating domain-specific, interdisciplinary working groups to develop reusable methods for linking legal obligations to technical practices. The authors emphasize that as we move toward general-purpose AI models, the burden of establishing compliance must shift toward understanding the specific, situated contexts where these technologies are actually deployed.

Summary updated

Abstract

The EU AI Act imposes technical and organizational requirements for high-risk AI systems, but applying these requirements involves contextual choices about what to measure and what evidence supports a compliance claim. We investigate these challenges through a participatory workshop involving 32 researchers, policymakers, and practitioners from different disciplines, using realistic scenarios in healthcare, recruitment, and public transport to examine how regulatory obligations are interpreted, translated into technical requirements, and evaluated in practice. Through a thematic analysis of workshop data, we identify recurring tensions between how regulatory obligations are translated into measurable system properties, how responsibility and human oversight are exercised in practice, and how the validity of compliance evidence is affected by changing operational conditions and system uses. For example, participants highlighted how fairness metrics could conceal discrimination in recruitment, how time pressure and limited information could undermine human oversight in public transport, and how changes in patient populations could invalidate previously established evidence of system performance. Our findings suggest that technical, organizational, and evidentiary challenges are interdependent, and that establishing and maintaining compliance requires attention not only to technical assessments but also to how AI systems are used, governed, and monitored over time. We discuss implications for supporting context-sensitive and ongoing compliance.
BibTeX
@misc{torkamaan2026between,
  author = {Helma Torkamaan and Masoud Mansoury and Yanan Xin and Oscar Oviedo Trespalacios and Roberta Savella and Andrej Zwitter and Maria Heuss and Huy Son Nguyen and Marie-Therese Sekwenz and Moyi Li and Roberta Tamponi and Megha Khosla},
  title = {{Between Obligations and Evidence: A Participatory Study of Translating the EU AI Act into Technical Requirements}},
  year = {2026},
  eprint = {2611.18439},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18439},
}
Figure