An Explainable Artificial Intelligence Software Tool for Weight Management Experts (PRIMO): Mixed Methods Study

Glenn Fernandes, Arthur Choi, Jacob M. Schauer, Angela Fidler Pfammatter, Bonnie Spring, Adnan Darwiche, Nabil AlshurafaJournal of Medical Internet Research 2023 · DOI

Abstract

Machine learning (ML) offers significant potential for early prediction of success in weight loss treatment programs, yet the "black box" nature of high-performing models often hinders clinical adoption. This study introduces PRIMO, an interactive explainable AI (XAI) tool designed to bridge the trust gap for weight management experts. By converting complex Random Forest models into tractable decision diagrams, PRIMO provides exact, logic-based explanations. Our results demonstrate that PRIMO significantly increases clinician agreement with AI-driven forecasts, enabling more timely and dynamic modifications to intervention strategies.

The Challenge: The Performance-Explainability Trade-off

In clinical weight management, "stepped-care" models—where interventions are adjusted early for non-responders—can optimize outcomes and reduce costs. While ML models can predict 6-month weight loss success with high accuracy using only two weeks of data, these models are frequently opaque. Clinicians are often hesitant to trust tools they cannot interpret, leading to a gap between research accuracy and clinical utility. Traditional XAI methods like LIME or SHAP provide local approximations that can be inconsistent or unfaithful to the underlying model logic.

The trade-off between model performance and explainability (A) and the PRIMO workflow for providing logic-based explanations to clinicians (B).

Methodology: From Black Box to Logic-Based Explanations

Researchers developed a multi-stage approach to create a faithful and interactive explanation system:

  1. Model Training: A Random Forest (RF) classifier was trained on data from the Opt-IN study ($N=419$). It utilized five key features collected via smartphone during the first two weeks of intervention: average daily food entry, custom food entry, weekly weight change, daily fat intake, and daily saturated fat intake.
  2. Knowledge Compilation: To move beyond approximations, the RF model was compiled into Ordered Binary Decision Diagrams (OBDDs). This allowed the system to compute Prime Implicants—the minimal set of feature conditions sufficient to guarantee the model's specific output.
  3. The PRIMO Interface: The tool provides an interactive dashboard where experts can adjust patient features via sliders. The interface highlights "safe ranges"; as long as a patient's metrics remain within these bounds, the model's prediction is guaranteed to stay the same.

Key Results and Insights

Enhanced Trust and Agreement

The underlying Random Forest model achieved 81% accuracy (86% specificity, 69% sensitivity). In a mixed-methods evaluation with 14 weight management experts, PRIMO significantly outperformed baseline explainability methods. Experts were more likely to agree with the ML model when using PRIMO ($\chi^2=7.9; P=.02$), with odds ratios for agreement reaching 2.52 and 3.95 compared to simpler ranking or probability-based methods.

Interactivity as a Catalyst for Understanding

The study revealed that the ability to "tweak" data was crucial. By observing how far a value could shift before a prediction flipped, experts gained a tangible sense of the model's boundaries and certainty. However, a "causal gap" remains: while experts understood what the model used to decide, they still sought biological or behavioral reasons why specific thresholds (e.g., fat intake levels) were predictive of failure.

Discussion and Future Directions

PRIMO demonstrates that "Local" (individual patient) explanations are most effective when they are exact rather than approximate. However, clinicians still demand "Global" context, such as overall system accuracy and historical performance, to build full trust.

Limitations of the current study include a small expert sample size and the functional nature of the logic, which does not inherently provide causal reasoning. Future work will focus on hybrid XAI approaches that combine local prime implicant logic with global performance metrics and raw data visualizations to provide a holistic clinical decision-support experience.

Summary updated

Abstract

Our results show the potential for weight management experts to agree with the ML-based early prediction of success in weight loss treatment programs, enabling timely and dynamic modification of intervention components to enhance intervention effectiveness. Our findings provide methods for advancing the understandability and trust of ML models among weight management experts.
BibTeX
@article{fernandes2023explainable,
  author = {Glenn Fernandes and Arthur Choi and Jacob M. Schauer and Angela Fidler Pfammatter and Bonnie Spring and Adnan Darwiche and Nabil Alshurafa},
  title = {{An Explainable Artificial Intelligence Software Tool for Weight Management Experts (PRIMO): Mixed Methods Study}},
  journal = {Journal of Medical Internet Research},
  year = {2023},
  doi = {10.2196/42047},
}
Figure