Perspectives on incorporating expert feedback into model updates

Valerie Chen, Umang Bhatt, Hoda Heidari, Adrian Weller, Ameet Talwalkar

TL;DR

This paper introduces a systematic taxonomy to bridge the communication gap between non-technical domain experts (like doctors or lawyers) and machine learning practitioners. By categorizing feedback into "domain-level" or "observation-level" and mapping it to specific model updates (dataset, loss function, or parameter space), the authors provide a framework for ensuring AI systems better align with human values and specialized expertise.

What Problem Does This Paper Address?

Machine learning models are increasingly deployed in high-stakes environments—healthcare, criminal justice, and policy-making—where technical performance (accuracy) isn't the only metric that matters. To be truly useful, these models must incorporate the nuanced "preferences" and "values" of domain experts.

However, a "translation gap" exists. While ML engineers can easily incorporate feedback from other engineers, they often struggle to turn a lawyer's qualitative concern or a doctor's conceptual insight into a mathematical update. Most existing "human-in-the-loop" research focuses on technical users or crowdworkers performing simple labeling tasks. This paper addresses the lack of a systematic approach for translating high-level, non-technical expertise into actionable model improvements.

Methods

The authors propose a Feedback-Update Taxonomy organized along two primary axes:

  1. Levels of Expert Feedback:
    • Domain-level: High-level conceptual feedback (e.g., "The model shouldn't rely on a patient's zip code").
  2. Observation-level: Feedback on specific data points (e.g., "This specific X-ray is mislabeled" or "This feature attribution is wrong for this case").
  3. Types of Model Updates:
    • Dataset Updates: Changing the training data ($D$) through augmentation, preprocessing, or reweighting.
  4. Loss Function Updates: Adding constraints or regularizers to the optimization objective ($\mathcal{L}$).
  5. Parameter Space Updates: Directly editing model weights or restricting the set of allowable models ($\Theta$).

The paper maps a wide range of existing research—from fairness and interpretability to active learning and robotics—into these six categories (e.g., Domain-to-Loss or Observation-to-Dataset).

Datasets

As a perspective and survey paper, this work does not introduce new datasets or report training results on specific benchmarks. Instead, it references and categorizes prior work that utilizes diverse datasets, including:

  • Medical diagnostics (e.g., CheXpert for chest radiographs).
  • Criminal justice (e.g., recidivism prediction/COMPAS).
  • Computer vision and Natural Language Processing benchmarks used in active learning and fairness research.

Key Insights and Ideas

  • The Practitioner as Translator: The role of the ML practitioner is evolving. Beyond just training models, they must act as "translators" who decide whether a piece of human feedback is best handled by changing the data, the loss function, or the model architecture.
  • Interchangeability of Updates: The authors highlight that the same expert goal can often be achieved through different technical paths. For example, a fairness goal could be met by reweighting the dataset or by adding a fairness constraint to the loss function. Choosing the "best" path depends on the cost of data collection versus the complexity of the math.
  • The Prompting Problem: The way a practitioner "prompts" an expert (e.g., open-ended questions vs. forced-choice options) significantly impacts the quality and "actionability" of the feedback.
  • Expert vs. Crowdworker: Unlike crowdworkers, domain experts have limited time and high-value knowledge. The paper argues for "expert-in-the-loop" strategies that prioritize high-impact updates over volume.

Key Results

The paper provides a qualitative synthesis of the field rather than quantitative benchmarks. Its primary "result" is the identification of under-explored areas in the taxonomy:

  • Observation-to-Loss Function and Domain-to-Parameter Space are identified as emerging areas where more research is needed to make updates intuitive for non-technical users.
  • The authors provide an Illustrative Example involving a lawyer providing feedback on a recidivism model, demonstrating how different scenarios (e.g., "the data is unrepresentative" vs. "this factor is unimportant") map to different cells in their taxonomy.

Limitations and Future Work

  • Conflicting Feedback: The current taxonomy focuses on a single expert. The authors note that aggregating feedback from multiple diverse experts—who may disagree—remains a major open challenge.
  • Measuring "Impact": There is no standard metric for what makes a model update "impactful" from an expert's perspective. Future work needs to define how to measure the "alignment" between an expert's mental model and the updated ML model.
  • Interface Design: The paper calls for "Participatory Innovation"—designing better user interfaces that allow experts to "see" model behavior in ways that naturally elicit the most useful types of feedback.
  • Scope: The framework is currently limited to supervised learning; extending this to reinforcement learning or unsupervised settings is a suggested next step.

Summary updated

Abstract

Machine learning (ML) practitioners are increasingly tasked with developing models that are aligned with non-technical experts' values and goals. However, there has been insufficient consideration on how practitioners should translate domain expertise into ML updates. In this paper, we consider how to capture interactions between practitioners and experts systematically. We devise a taxonomy to match expert feedback types with practitioner updates. A practitioner may receive feedback from an expert at the observation- or domain-level, and convert this feedback into updates to the dataset, loss function, or parameter space. We review existing work from ML and human-computer interaction to describe this feedback-update taxonomy, and highlight the insufficient consideration given to incorporating feedback from non-technical experts. We end with a set of open questions that naturally arise from our proposed taxonomy and subsequent survey.
BibTeX
@article{chen2023perspectives,
  author = {Valerie Chen and Umang Bhatt and Hoda Heidari and Adrian Weller and Ameet Talwalkar},
  title = {{Perspectives on incorporating expert feedback into model updates}},
  journal = {Patterns},
  year = {2023},
  doi = {10.1016/j.patter.2023.100780},
}
Figure