TL;DR
This paper introduces a systematic taxonomy to bridge the communication gap between non-technical domain experts (like doctors or lawyers) and machine learning practitioners. By categorizing feedback into "domain-level" or "observation-level" and mapping it to specific model updates (dataset, loss function, or parameter space), the authors provide a framework for ensuring AI systems better align with human values and specialized expertise.
What Problem Does This Paper Address?
Machine learning models are increasingly deployed in high-stakes environments—healthcare, criminal justice, and policy-making—where technical performance (accuracy) isn't the only metric that matters. To be truly useful, these models must incorporate the nuanced "preferences" and "values" of domain experts.
However, a "translation gap" exists. While ML engineers can easily incorporate feedback from other engineers, they often struggle to turn a lawyer's qualitative concern or a doctor's conceptual insight into a mathematical update. Most existing "human-in-the-loop" research focuses on technical users or crowdworkers performing simple labeling tasks. This paper addresses the lack of a systematic approach for translating high-level, non-technical expertise into actionable model improvements.
Methods
The authors propose a Feedback-Update Taxonomy organized along two primary axes:
- Levels of Expert Feedback:
- Domain-level: High-level conceptual feedback (e.g., "The model shouldn't rely on a patient's zip code").
- Observation-level: Feedback on specific data points (e.g., "This specific X-ray is mislabeled" or "This feature attribution is wrong for this case").
- Types of Model Updates:
- Dataset Updates: Changing the training data ($D$) through augmentation, preprocessing, or reweighting.
- Loss Function Updates: Adding constraints or regularizers to the optimization objective ($\mathcal{L}$).
- Parameter Space Updates: Directly editing model weights or restricting the set of allowable models ($\Theta$).
The paper maps a wide range of existing research—from fairness and interpretability to active learning and robotics—into these six categories (e.g., Domain-to-Loss or Observation-to-Dataset).
Datasets
As a perspective and survey paper, this work does not introduce new datasets or report training results on specific benchmarks. Instead, it references and categorizes prior work that utilizes diverse datasets, including:
- Medical diagnostics (e.g., CheXpert for chest radiographs).
- Criminal justice (e.g., recidivism prediction/COMPAS).
- Computer vision and Natural Language Processing benchmarks used in active learning and fairness research.
Key Insights and Ideas
- The Practitioner as Translator: The role of the ML practitioner is evolving. Beyond just training models, they must act as "translators" who decide whether a piece of human feedback is best handled by changing the data, the loss function, or the model architecture.
- Interchangeability of Updates: The authors highlight that the same expert goal can often be achieved through different technical paths. For example, a fairness goal could be met by reweighting the dataset or by adding a fairness constraint to the loss function. Choosing the "best" path depends on the cost of data collection versus the complexity of the math.
- The Prompting Problem: The way a practitioner "prompts" an expert (e.g., open-ended questions vs. forced-choice options) significantly impacts the quality and "actionability" of the feedback.
- Expert vs. Crowdworker: Unlike crowdworkers, domain experts have limited time and high-value knowledge. The paper argues for "expert-in-the-loop" strategies that prioritize high-impact updates over volume.
Key Results
The paper provides a qualitative synthesis of the field rather than quantitative benchmarks. Its primary "result" is the identification of under-explored areas in the taxonomy:
- Observation-to-Loss Function and Domain-to-Parameter Space are identified as emerging areas where more research is needed to make updates intuitive for non-technical users.
- The authors provide an Illustrative Example involving a lawyer providing feedback on a recidivism model, demonstrating how different scenarios (e.g., "the data is unrepresentative" vs. "this factor is unimportant") map to different cells in their taxonomy.
Limitations and Future Work
- Conflicting Feedback: The current taxonomy focuses on a single expert. The authors note that aggregating feedback from multiple diverse experts—who may disagree—remains a major open challenge.
- Measuring "Impact": There is no standard metric for what makes a model update "impactful" from an expert's perspective. Future work needs to define how to measure the "alignment" between an expert's mental model and the updated ML model.
- Interface Design: The paper calls for "Participatory Innovation"—designing better user interfaces that allow experts to "see" model behavior in ways that naturally elicit the most useful types of feedback.
- Scope: The framework is currently limited to supervised learning; extending this to reinforcement learning or unsupervised settings is a suggested next step.
Summary updated