Data Efficient Paradigms for Personalized Assessment of Black-Box Taskable AI Systems

Pulkit Verma

Abstract

The vast diversity of internal designs of taskable black-box AI systems and their nuanced zones of safe functionality make it difficult for a layperson to use them without unintended side effects. My dissertation focuses on developing paradigms that enable a user to assess and understand the limits of an AI system's safe operability. We develop a personalized AI assessment module that lets an AI system execute instruction sequences in simulators and answer queries about these executions. Our results show that such a primitive query-response interface is sufficient to efficiently derive a user-interpretable model of a system's capabilities.
BibTeX
@inproceedings{verma2024data,
  author = {Pulkit Verma},
  title = {{Data Efficient Paradigms for Personalized Assessment of Black-Box Taskable AI Systems}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30414},
}

Loading related content...

Figure