Extreme Multilabel Classification for Specialist Doctor Recommendation with Implicit Feedback and Limited Patient Metadata

Filipa Valdeira, Stevo Rackovic, Valeria Danalachi, Qiwei Han, Cláudia SoaresarXiv 2023 · DOI · arXiv

Recommending specialist doctors to patients is a critical healthcare challenge, often hindered by sparse patient data and the "cold-start" problem for new users. This research proposes a novel approach by reframing doctor recommendation as an Extreme Multilabel Classification (XML) task. Using the DECAF model, which learns joint embeddings for patients and doctors in a shared feature space, the researchers demonstrate significant performance gains over traditional recommendation systems.

The Recommendation Challenge

Primary care physicians (PCPs) often face challenges in referring patients to the most appropriate specialists due to limited consultation time and incomplete knowledge of vast provider networks. Digital recommendation systems (RS) offer a potential solution but face three major hurdles:

  1. The Cold-Start Problem: Recommending doctors to patients the system has never seen before.
  2. Implicit Feedback: Visit history provides only binary data (visited or not) rather than explicit quality ratings.
  3. Privacy and Metadata: Strict regulations like GDPR limit the available patient metadata to basic demographics and location.

Methodology: XML and DECAF

The authors treat doctor recommendation as a Multilabel Classification problem where patients are instances and doctors are labels. Given the thousands of possible doctors, this is categorized as "Extreme" (XML). They utilize DECAF (Deep Extreme Classification with Label Features), a model that embeds both patient features and doctor labels into a unified vector space.

To evaluate the impact of different information types, the researchers defined five experimental scenarios (S-1 to S-5) based on combinations of patient (P) and doctor (D) metadata, including demographics, medical education, specialization history, and hospital proximity.

Table 1: Feature availability across experimental scenarios S-1 to S-5 for Patients (P) and Doctors (D).

Key Results

The XML approach was compared against industry-standard recommendation models, including SVD, BiVAE, and LightFM, using a massive real-world dataset of 1 million patients and 5.5 million interactions from a European healthcare provider.

Figure 1: Performance comparison of DECAF against baseline models for seen and new patients across PS-nDCG@3 and Recall@10 metrics.

Performance Insights

  • Superiority for Seen Patients: DECAF outperformed the strongest baseline (LightFM), improving PS-nDCG@3 (a measure of ranking quality) from 25.36 to 27.18 when all features were available.
  • Cold-Start Robustness: The performance gap widened significantly for new patients. DECAF achieved a Recall@10 of 23.10, while LightFM trailed at 15.85. Traditional methods like SVD failed almost entirely in this scenario.
  • Predictive Power of Proximity: For new patients without medical history, physical distance to hospitals (Scenario 4) proved to be the most effective predictor of specialist choice.
  • Diversity in Recommendations: The XML framework proved more effective at recommending "tail" doctors—specialists who are less popular but may be the ideal fit—rather than defaulting to the most visited providers.

Conclusion

By reframing recommendation as an XML task, this research provides a robust framework for specialist referrals that excels even when patient data is sparse. The ability of the DECAF model to leverage both patient and doctor features in a shared space allows it to overcome traditional collaborative filtering limitations, particularly for the critical cold-start problem in healthcare.

Summary updated

Abstract

Recommendation Systems (RS) are often used to address the issue of medical doctor referrals. However, these systems require access to patient feedback and medical records, which may not always be available in real-world scenarios. Our research focuses on medical referrals and aims to predict recommendations in different specialties of physicians for both new patients and those with a consultation history. We use Extreme Multilabel Classification (XML), commonly employed in text-based classification tasks, to encode available features and explore different scenarios. While its potential for recommendation tasks has often been suggested, this has not been thoroughly explored in the literature. Motivated by the doctor referral case, we show how to recast a traditional recommender setting into a multilabel classification problem that current XML methods can solve. Further, we propose a unified model leveraging patient history across different specialties. Compared to state-of-the-art RS using the same features, our approach consistently improves standard recommendation metrics up to approximately $10\%$ for patients with a previous consultation history. For new patients, XML proves better at exploiting available features, outperforming the benchmark in favorable scenarios, with particular emphasis on recall metrics. Thus, our approach brings us one step closer to creating more effective and personalized doctor referral systems. Additionally, it highlights XML as a promising alternative to current hybrid or content-based RS, while identifying key aspects to take into account when using XML for recommendation tasks.
BibTeX
@misc{valdeira2023extreme,
  author = {Filipa Valdeira and Stevo Rackovic and Valeria Danalachi and Qiwei Han and Cláudia Soares},
  title = {{Extreme Multilabel Classification for Specialist Doctor Recommendation with Implicit Feedback and Limited Patient Metadata}},
  year = {2023},
  eprint = {2308.11022},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.11022},
}
Figure