Making the Right Thing: Bridging HCI and Responsible AI in Early-Stage AI Concept Selection

Ji-Youn Jung, Devansh Saxena, Minjung Park, Jini Kim, Jodi Forlizzi, Kenneth Holstein, John Zimmerman

Introduction

Artificial Intelligence projects face a staggering 85% failure rate. A primary driver of this trend is that ethical and commercial risks are frequently addressed only after a model has been built—a "reactive" approach to Responsible AI (RAI). This paper proposes a shift from "making the thing right" (technical optimization) to "making the right thing" (selecting viable, ethical concepts). By integrating RAI principles into the earliest stages of innovation—concept selection—teams can identify high-value, low-risk opportunities before investing in development.

The Design-Led Framework

The researchers employed a Research through Design (RtD) approach to develop a suite of tools aimed at bridging the gap between HCI and AI engineering. The framework centers on two primary artifacts:

  1. Concept Cards: These templates deconstruct AI ideas into "Do-Reason-Know" components (actions, inferences, and data) while identifying four critical stakeholders: the Payer, End-User, Servicing Party, and Impacted Individual.
  2. Risk-Benefit Matrix: A 2x2 collaborative sorting tool used to visualize the trade-offs between potential value and ethical/technical risks.
Figure 1: A Concept Card for a hypothetical Teacher Evaluation System, detailing stakeholders, system actions, and data requirements.

To test these tools, the researchers conducted a 90-minute workshop with 15 industry professionals, including UX designers, data scientists, and product managers. Participants evaluated six hypothetical AI concepts in the education sector, ranging from AI proctors to storytelling assistants. The workshop utilized a structured probe to guide participants through individual ratings and team-based consensus building.

Figure 2: The workshop probe layout, showing the transition from initial concept description to multi-dimensional risk/benefit rating and final matrix placement.

Key Insights: RAI as a Commercial Filter

The study revealed that practitioners do not view RAI as a mere "compliance bolt-on." Instead, ethical concerns served as a commercial filter. For instance, a teacher evaluation system was deemed unviable not just because it was "unethical," but because the high risk of alienating teachers would undermine the product's fundamental utility.

The Concept Card emerged as the most valued tool. By forcing teams to specify exactly what data is used and what inferences are made, it stripped away the "magic" of AI, enabling realistic technical and ethical critiques. Interestingly, while teams reached consensus on risks quickly, they spent significant time debating benefits. This highlighted a recurring tension: "Benefit for whom?" Often, what benefits a "payer" (e.g., a school district) may negatively impact the "end-user" (e.g., a student).

Results and Impact

The structured process demonstrated remarkable consistency. Despite working in ad-hoc groups, all four teams independently identified the same "low-risk, high-benefit" and "high-risk, low-benefit" concepts. Furthermore, 100% of participants found the framework more rigorous than their current industry practices, with several expressing immediate intent to adopt Concept Cards in their organizations.

The framework also facilitated "reframing" rather than simple rejection. One team, for example, transformed a high-stakes automated grading tool into a "teacher companion," effectively lowering the ethical stakes while maintaining the core value proposition.

Conclusion and Future Work

While the study successfully demonstrated the utility of these tools for "Visioning" new products, future research is needed to see how they perform within established teams where power dynamics and shared histories might influence decision-making. Additionally, testing the framework on "Versioning" (incremental updates to existing systems) and on teams' own "darling" ideas will provide further insight into how to best bridge the gap between technical ambition and responsible innovation.

Summary updated

BibTeX
@inproceedings{jung2025making,
  author = {Ji-Youn Jung and Devansh Saxena and Minjung Park and Jini Kim and Jodi Forlizzi and Kenneth Holstein and John Zimmerman},
  title = {{Making the Right Thing: Bridging HCI and Responsible AI in Early-Stage AI Concept Selection}},
  booktitle = {Conference on Designing Interactive Systems},
  year = {2025},
  doi = {10.1145/3715336.3735745},
}
Figure