Sample Efficient Paradigms for Personalized Assessment of Taskable AI Systems

Pulkit Verma

Abstract

As autonomous AI systems increasingly perform complex sequential tasks, their internal logic often remains opaque to human users. This lack of transparency can lead to unsafe usage and unintended side effects. This paper presents a Personalized AI Assessment Module (AAM) designed to bridge the gap between complex AI internal states and human-understandable concepts. By framing query generation as a planning problem and utilizing top-down action model inference, the AAM efficiently interrogates black-box systems to build interpretable models of their capabilities.

Introduction

The deployment of autonomous agents in real-world environments—ranging from household robots to complex software agents—necessitates that users understand the limits and capabilities of these systems. However, a significant "knowledge gap" exists: lay users often do not know which questions to ask to verify safety, and a "vocabulary gap" persists where the AI's internal representations do not align with human concepts.

Existing approaches to capability assessment often rely on passive observation, which is inherently slow and prone to learning "buggy" or incomplete models. This research proposes an active interrogation-based framework that treats the assessment process as a sample-efficient planning problem.

Methodology: The Personalized AI Assessment Module (AAM)

The core contribution of this work is the Personalized AI Assessment Module (AAM). Unlike passive systems, the AAM actively probes the black-box AI to deduce its underlying logic.

Architecture of the Personalized AI Assessment Module (AAM).

The AAM operates through several key mechanisms:

Active Interrogation Policies

The module reduces the challenge of "asking the right question" to a formal planning problem. It iteratively generates queries based on the AI's previous responses, systematically narrowing down the space of possible models until the AI's true capabilities are identified.

Top-Down Action Model Inference

To manage the combinatorial explosion of potential models, the AAM employs a top-down approach. It tests pairs of abstract models; if an abstract model fails a test, the system can immediately prune an entire branch of more detailed refinements. This hierarchical approach significantly improves sample efficiency.

Capability Discovery

The system distinguishes between primitive actions (e.g., "move left") and high-level capabilities (e.g., "navigate to a room"). By observing state transitions during interrogation, the AAM learns to describe these high-level behaviors in a way that aligns with the user's vocabulary.

Evaluation and Results

The framework was evaluated through several iterations—AIA, DAAISy, and QACE—using standard International Planning Competition (IPC) domains and human user studies.

Performance Benchmarks

  • AIA (Agent Interrogation Algorithm): Demonstrated faster convergence to correct models compared to the state-of-the-art FAMA algorithm in deterministic environments.
  • DAAISy (Differential Assessment): Proved capable of updating models for evolving AI systems without requiring a full restart of the assessment process.
  • QACE (Query-based Autonomous Capability Estimation): In stochastic environments, QACE achieved high sample efficiency and "few-shot generalization," outperforming the GLIB baseline.

User Study Findings

A behavioral analysis study confirmed the human-centric benefits of the AAM. Participants presented with high-level "capability descriptions" were significantly faster and more accurate in assessing the AI's limits than those provided with low-level primitive action sequences.

Discussion and Future Work

The research demonstrates that active interrogation is superior to passive observation for learning accurate models of black-box AI. Crucially, the AI system does not need to be pre-programmed with the user's vocabulary; the AAM serves as an effective translator.

Current limitations include the assumption of full observability in the environment. Future work will focus on extending the framework to partially observable settings and integrating these modules into assistive AI systems to create agents that are inherently compliant and understandable to their human counterparts.

Summary updated

Abstract

The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones of safe functionality make it difficult for a layperson to use them without unintended side effects. The focus of my dissertation is to develop paradigms that would enable a user to assess and understand the limits of an AI system's safe operability. We develop a personalized AI assessment module that lets an AI system execute instruction sequences in simulators and answer queries about these executions. Our results show that such a primitive query-response capability is sufficient to efficiently derive a user-interpretable model of the system's capabilities in fully observable settings.
BibTeX
@inproceedings{verma2023sample,
  author = {Pulkit Verma},
  title = {{Sample Efficient Paradigms for Personalized Assessment of Taskable AI Systems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.24963/ijcai.2023/824},
}
Figure