Assessing Ethical Thinking about AI

Judy Goldsmith, Emanuelle Burton, David M. Dueber, Beth Goldstein, Shannon Sampson, Michael D. TolandAAAI 2020 · DOI

Introduction

As artificial intelligence increasingly shapes the fabric of modern society, universities have responded by integrating ethics into computer science curricula. However, while the "surge" in AI ethics education is well-documented, a critical gap remains: the community lacks validated tools to measure whether students are actually developing the capacity for ethical thought. In the position paper "Assessing Ethical Thinking about AI," presented at AAAI-20, the authors argue that current assessment methods are insufficient for the unique challenges of the AI field. They propose a shift away from traditional "right vs. wrong" testing toward a framework that prioritizes a student's ability to describe complex ethical landscapes and their internal sense of professional responsibility.

The Measurement Gap in AI Ethics

The rapid expansion of AI ethics courses has outpaced the development of standardized evaluation metrics. Currently, instructors often rely on traditional exams or essays, which are difficult to validate or compare across institutions. Furthermore, existing standardized tools, such as the Defining Issues Test (DIT-2), are frequently criticized for being too generic.

The authors contend that tests like the DIT-2 are often rooted in specific philosophical frameworks (such as deontology) and focus primarily on "moral judgment"—the act of ranking pre-defined solutions to a problem. This approach fails to capture the reality of AI development, where the most pressing ethical issues are often "invisible" at the outset. For an AI professional, the primary challenge is not just choosing the right answer from a list, but recognizing that an ethical dilemma exists within a technical design in the first place.

A Proposed Framework for Assessment

To address these shortcomings, the paper outlines a conceptual framework for a new assessment tool built on two primary facets:

1. Ethical Description

This facet focuses on the cognitive ability to identify, articulate, and analyze the costs, benefits, and stakeholders associated with a new technology. Unlike normative ethics, which seeks to determine what should be done, ethical description is about seeing the landscape clearly. It involves mapping out the potential ramifications of a technology before it is released, ensuring that developers can anticipate unintended consequences.

2. Felt Ethical Responsibility

The second facet is affective rather than cognitive. Drawing on Platonic philosophy—which suggests that a true understanding of "the good" is inextricably linked to the desire to be good—the authors argue that assessment must also account for a student’s internal commitment to act on their ethical insights. This "felt responsibility" is what bridges the gap between recognizing a problem and taking action to mitigate it.

Key Insights: Sensitivity and Pedagogy

A central theme of the paper is that "assessment drives instruction." Invoking Cronbach’s principle that what an evaluator measures tends to become the primary goal of a program, the authors argue that by creating better assessment tools, the AI community can steer instruction toward more meaningful outcomes.

One such outcome is "ethical sensitivity"—the ability to detect emergent dilemmas in a rapidly shifting technological landscape. To cultivate this, the authors highlight the use of science fiction in the classroom. By exploring the societal impacts of fictional technologies, students can practice "descriptive insight" in a low-stakes environment, honing their ability to foresee the complex interactions between code and society.

Conclusion and Future Directions

As a "blue sky" proposal, this paper does not offer a finished tool or quantitative data. Instead, it serves as a roadmap for future interdisciplinary collaboration. The authors call for AI researchers to partner with psychometricians (experts in test and measurement) to define "ethical reasoning skill" operationally and build a tool that includes "constructed response" (open-ended) elements.

The ultimate goal is a validated, discipline-specific instrument that can evaluate the effectiveness of different teaching methods and ensure that the next generation of AI professionals is prepared for the workforce. By moving beyond multiple-choice recognition toward a deeper evaluation of descriptive and affective capacities, the field can better ensure that ethical thinking becomes a core competency of AI engineering.

Summary updated

BibTeX
@inproceedings{goldsmith2020assessing,
  author = {Judy Goldsmith and Emanuelle Burton and David M. Dueber and Beth Goldstein and Shannon Sampson and Michael D. Toland},
  title = {{Assessing Ethical Thinking about AI}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i09.7075},
}

Loading related content...

Figure