Trust, Resilience and Interpretability of AI Models

Susmit Jha

Abstract

In this tutorial, we present our recent work on building trusted, resilient and interpretable AI models by combining symbolic methods developed for automated reasoning with connectionist learning methods that use deep neural networks. The increasing adoption of artificial intelligence and machine learning in systems, including safety-critical systems, has created a pressing need for developing scalable techniques that can be used to establish trust over their safe behavior, resilience to adversarial attacks, and interpretability to enable human audits. This tutorial is comprised of three components: review of techniques for verification of neural networks, methods for using geometric invariants to defend against adversarial attacks, and techniques for extracting logical symbolic rules by reverse engineering machine learning models. These techniques form the core of TRINITY: Trusted, Resilient and Interpretable AI framework being developed at SRI. In this tutorial, we identify the key challenges in building the TRINITY framework, and report recent results on each of these three fronts.
BibTeX
@inproceedings{jha2019trust,
  author = {Susmit Jha},
  title = {{Trust, Resilience and Interpretability of AI Models}},
  booktitle = {NSV@CAV},
  year = {2019},
  doi = {10.1007/978-3-030-28423-7\_1},
}

Loading related content...

Figure