Unifying Principles and Metrics for Safe and Assistive AI

Siddharth SrivastavaAAAI 2021 · OpenReview

Abstract

The prevalence and success of AI applications have been tempered by concerns about the controllability of AI systems and AI's impact on the future of work. These concerns reflect two aspects of a central question: how would humans work with AI systems? While research on AI safety focuses on designing AI systems that allow humans to safely instruct and control AI systems, research on AI and the future of work focuses on the impact of AI on humans who may be unable to do so. This Blue Sky Ideas paper proposes a unifying set of declarative principles that enable a more uniform evaluation of arbitrary AI systems along multiple dimensions of the extent to which they are suitable for use by specific classes of human operators. It leverages recent AI research and the unique strengths of the field to develop human-centric principles for AI systems that address the concerns noted above.

Introduction

As AI systems transition from controlled laboratory environments to the complexities of the real world, two primary anxieties have emerged: the technical challenge of AI safety (maintaining control) and the socio-economic challenge of the future of work (the potential displacement of non-technical workers). Traditionally, these issues have been treated as separate domains. However, they are fundamentally linked by the human-AI relationship. This paper argues for a shift in how we evaluate AI, moving away from internal performance metrics toward "declarative principles" that measure how well a system serves its human operators.

The Limitations of Current Metrics

Most current AI research relies on "Level 0" metrics, such as computational speed, data requirements, and accuracy. While these are essential for benchmarking algorithms, they are insufficient for assessing real-world impact because they ignore the human in the loop. Without human-centric metrics, we cannot objectively determine if an AI is "safe" for a factory worker to operate or if it requires a PhD-level expert to prevent catastrophic failure. To bridge this gap, we need a standardized way to compare AI architectures based on their usability, safety, and the burden they place on the human user.

A Hierarchy of Principles for Taskable AI

This paper introduces a conceptual framework for "Taskable AI"—systems capable of performing high-level tasks like household robotics or digital assistance—organized into a hierarchy of principles:

Level I: Principle of Safe Use (P1)

"An AI system must make it easy for its operators to use it safely." This principle focuses on the immediate interaction and is evaluated across four dimensions:

  • Inclusivity (E1): The breadth of the potential operator set (e.g., experts vs. the general public).
  • Ease of Instruction (E2): The difficulty involved in modifying the AI’s behavior or assigning new tasks.
  • Task Scope (E3): The range of tasks the system can actually perform.
  • Safety Guarantees (E4): The system’s ability to provide rigorous upper bounds on risk.

Level II: Principle of Safe Learning (P2)

"An AI system must make it easy for its operators to learn how to use it safely." This principle addresses the "Training Blind Spot." A system might appear safe under P1 only because its operators have undergone years of specialized training. P2 ensures that the cost of training is itself a metric of AI quality:

  • Training Cost (E5): The prerequisites and resources required to train a human to operate the system safely, including the ability to learn "on the job."

Key Insights and Strategic Implications

Closing the Training Loophole

By making training cost (E5) a core metric, the framework prevents a "safety loophole" where unsafe or complex systems claim safety by restricting their use to a tiny, elite group of experts. To truly democratize AI, we must find ways to maintain high safety guarantees (E4) while expanding the operator set (E1) to non-experts.

Recontextualizing Explainable AI (XAI)

In this framework, XAI is no longer just a tool for building trust; it is a functional necessity for satisfying Principle P2. Effective explanations lower the barrier for humans to learn how to safely command the machine, directly reducing the training cost and increasing the inclusivity of the system.

Independent Evaluation and the Minimal Interface

The paper advocates for a declarative approach—evaluating systems based on external outcomes rather than internal code. It suggests that AI systems should support a "minimal query-response interface." This would allow third parties to conduct "black-box" testing by asking high-level questions (e.g., "In what situations can you perform this task?") to map capabilities without needing access to proprietary logic.

Conclusion and Future Work

The formalization of the five dimensions (E1–E5) provides a roadmap for future AI development. However, several challenges remain. The primary hurdle is operationalization: turning these intuitive principles into quantifiable, mathematical metrics. Furthermore, this shift requires an interdisciplinary bridge between AI research, human factors engineering, and industrial safety. By focusing on these human-centric principles, we can ensure that the next generation of AI is not only powerful but also truly assistive and inclusive.

Summary updated

Abstract

The prevalence and success of AI applications have been tempered by concerns about the controllability of AI systems about AI's impact on the future of work. These concerns reflect two aspects of a central question: how would humans work with AI systems? While research on AI safety focuses on designing AI systems that allow humans to safely instruct and control AI systems, research on AI and the future of work focuses on the impact of AI on humans who may be unable to do so. This Blue Sky Ideas paper proposes a unifying set of declarative principles that enable a more uniform evaluation of arbitrary AI systems along multiple dimensions of the extent to which they are suitable for use by specific classes of human operators. It leverages recent AI research and the unique strengths of the field to develop human-centric principles for AI systems that address the concerns noted above.
BibTeX
@inproceedings{srivastava2021unifying,
  author = {Siddharth Srivastava},
  title = {{Unifying Principles and Metrics for Safe and Assistive AI}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=npph8pp4IFL},
}
Figure