Towards Explainable and Trustworthy Autonomous Physical Systems

Daniel Omeiza, Sule Anjomshoae, Konrad Kollnig, Oana-Maria Camburu, Kary Främling, Lars KunzeCHI Extended Abstracts 2021 · DOI

TL;DR

This paper proposes a workshop focused on making autonomous physical systems (like robots and self-driving cars) more understandable and reliable. The goal is to bring together researchers and practitioners to discuss the challenges and opportunities in designing systems that can explain their actions, fostering greater public trust and safer integration into society.

What Problem Does This Paper Address?

As autonomous physical systems become more prevalent, especially in critical areas like healthcare and transportation, ensuring their safety and public acceptance is paramount. Current "black-box" systems, where the decision-making process is opaque, hinder assessment by designers and regulators, making improvements and building trust difficult. This lack of transparency can lead to serious consequences, such as a pedestrian misinterpreting a self-driving car's action. The paper addresses the need for these systems to be explainable and trustworthy, enabling them to communicate their reasoning, justify their actions, and signal unstable conditions, thereby improving human-system collaboration and facilitating error correction.

Methods

This paper doesn't describe a specific technical method or algorithm. Instead, it outlines a proposal for a one-day virtual workshop titled "Towards Explainable and Trustworthy Autonomous Physical Systems (ETAPS)." The workshop's methodology involves:

  • Pre-workshop Survey: To gauge public perception of autonomous systems' trustworthiness and identify critical issues.
  • Panel Discussions: To foster interaction and address challenges identified in the survey and talks.
  • Spotlight Talks: Short presentations from invited speakers and contributors.
  • Keynotes: Presentations from experts across various fields.
  • Post-workshop Report: A summary of findings, recommendations, and survey results.
  • Virtual Format: To ensure global accessibility and maximize participation.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • The "Black-Box" Problem: The core issue is that many advanced autonomous systems operate as "black boxes," making it hard for humans to understand why they do what they do.
  • Trust is Crucial for Adoption: Public acceptance and adoption of autonomous systems hinge on users trusting them. This trust is directly impacted by the system's explainability.
  • Explainability Enhances Collaboration: When autonomous systems can explain their actions, it facilitates better collaboration with humans, leading to improved performance and common understanding.
  • Beyond Technical: Social and Ethical Dimensions: The workshop emphasizes that addressing explainability and trustworthiness requires input from social scientists, ethicists, and human-computer interaction experts, not just AI engineers.
  • Proactive vs. Reactive Measures: The paper highlights the need for systems to not only explain their normal operations but also to communicate unstable conditions or potential adversarial attacks.
  • Global Reach and Accessibility: The workshop is designed to be virtual to overcome geographical barriers and encourage broad participation.

Key Results

As this paper proposes a workshop and not a research study with empirical results, there are no quantitative or qualitative results to summarize. The "results" are the intended outcomes of the workshop itself, which include:

  • Establishing common ground for the study and development of human-centered explainable and trustworthy autonomous physical systems.
  • Exploring the components of trustworthy autonomous physical systems.
  • Assessing the impact of explanations on user confidence, understandability, and trust.
  • Discussing applications and contributions towards overcoming the lack of explainability and trust.
  • Discussing alternative means for the next generation of explainable and trustworthy autonomous physical systems.

Limitations and Future Work

The paper itself doesn't present limitations of a study, but rather outlines the scope and goals of the proposed workshop. The future work is implicitly the successful execution of the workshop and the dissemination of its findings through the technical report and website. The workshop aims to identify and address the challenges and opportunities in the field, paving the way for future research and development in explainable and trustworthy autonomous physical systems.

Summary updated

BibTeX
@inproceedings{omeiza2021towards,
  author = {Daniel Omeiza and Sule Anjomshoae and Konrad Kollnig and Oana-Maria Camburu and Kary Främling and Lars Kunze},
  title = {{Towards Explainable and Trustworthy Autonomous Physical Systems}},
  booktitle = {CHI Extended Abstracts},
  year = {2021},
  doi = {10.1145/3411763.3441338},
}
Figure