Towards explained treatment search results: feature analysis and explanation formulation

Edeline Contempré, Zoltán Szlávik, Erick Godinez, Annette ten Teije, Ilaria TiddiDigital 2021

TL;DR

This paper tackles the challenge of increasing healthcare professionals' (HCPs) trust in AI-powered treatment search engines. They propose a system that re-orders search results based on explainability and provides user-friendly explanations for each document, aiming to make AI outcomes more understandable and trustworthy for medical decision-making.

What Problem Does This Paper Address?

Healthcare professionals increasingly rely on AI tools to find relevant clinical trials for patients, especially those with limited treatment options. However, as AI models become more complex, their decision-making processes become opaque, leading to a lack of trust among users. This is particularly critical in medicine, where errors can have severe consequences. The paper addresses this trust deficit by proposing a method to make AI-driven treatment search results more transparent and understandable to HCPs.

Methods

The core of the proposed method involves generating "local explainability scores" for each search result. This is achieved through a pipeline that includes:

  • Feature Engineering: Identifying and creating user-interpretable features that describe clinical trials. These features are categorized as either "query dependent" (e.g., "Query in title") or "query independent" (e.g., "Number of publications").
  • Local Explainability Scoring: Assigning scores to each retrieved trial based on these features.
  • Re-ordering of Results: Using these explainability scores to re-order the search results, prioritizing those that are more interpretable.
  • Explanation Formulation: Generating user-friendly, natural language sentences that explain why a particular trial is relevant, based on the identified features.

A key aspect of their approach is the use of crowdsourcing to determine the importance of different features for explainability and to assess user preferences for how explanations are formulated. This allows them to gather insights from a broader audience, which they hypothesize can yield quality comparable to expert annotations.

Datasets

The paper does not explicitly mention specific datasets used for training or evaluation in the traditional sense. Instead, they leverage:

  • Clinical trial registries: Primarily clinicaltrials.gov, and potentially others, to extract trial information.
  • Medical knowledge graphs: Such as UMLS and Disease Ontology, to aid in understanding medical conditions and their relationships.
  • PubMed abstracts: Used for some feature extraction.
  • Crowdsourced data: Collected via Amazon Mechanical Turk (using Amazon SageMaker Ground Truth) for assessing feature importance and explanation formulation preferences. This crowdsourced dataset is a novel contribution for this specific research context.

Key Insights and Ideas

  • Explainability as a Trust-Builder: The fundamental idea is that making the AI's reasoning transparent is crucial for building trust, which in turn encourages adoption and effective use of AI tools in healthcare.
  • User-Centric Feature Importance: Relying on crowdsourcing to gauge feature importance for explainability is a pragmatic approach, especially when direct user interaction data is limited. It acknowledges that what makes an explanation "good" is subjective and user-dependent.
  • Formulation Matters: The way explanations are phrased significantly impacts their perceived usefulness. The study highlights preferences for non-numeric, fact-driven, and potentially disease-specific explanations.
  • Intuitive Features are Less Convincing: Surprisingly, features that are intuitively obvious to users (like "Query in title" or "Trial is recruiting") might be less convincing as explanations because they don't offer new insights.

Key Results

The crowdsourcing study yielded several key findings:

  • Feature Importance: "Query in detailed description" was found to be the most convincing feature (mean score of 3.69 on a 5-point Likert scale), while "Query in title" and "Trial is recruiting" were the least convincing (scores of 3.15 and 3.13, respectively).
  • Explanation Formulation Preferences:
    • Users prefer non-numeric explanations (e.g., "multiple times" over "5 times").
  • Users prefer fact-driven explanations (e.g., "clearly mentioned") over action-oriented ones (e.g., "retrieved").
  • There was no significant preference difference between specifying the disease (e.g., "HIV") versus using a general term ("the condition").

These results provide guidance for feature weighting in the re-ordering model and for formulating explanatory sentences.

Limitations and Future Work

  • Limitations:
    • The study presents "preliminary results" and is described as a "work-in-progress project."
    • No real-time user interaction data was available, necessitating the use of crowdsourcing.
  • Lay participants were not asked to review actual clinical trial documents due to their complexity.
  • Future Work:
    • Implement an "explanation engine" and measure its impact on re-ordering and explanations.
  • Address the "cold start problem" for explanation engines that learn through user interaction, if privacy policies allow.
  • Further research is needed on how search experiences, particularly in the medical domain, should be designed to best meet user needs for explanations.

Summary updated

BibTeX
@inproceedings{contempre2021towards,
  author = {Edeline Contempré and Zoltán Szlávik and Erick Godinez and Annette ten Teije and Ilaria Tiddi},
  title = {{Towards explained treatment search results: feature analysis and explanation formulation}},
  booktitle = {Digital},
  year = {2021},
}
Figure