Recommending specialist doctors to patients is a critical healthcare challenge, often hindered by sparse patient data and the "cold-start" problem for new users. This research proposes a novel approach by reframing doctor recommendation as an Extreme Multilabel Classification (XML) task. Using the DECAF model, which learns joint embeddings for patients and doctors in a shared feature space, the researchers demonstrate significant performance gains over traditional recommendation systems.
The Recommendation Challenge
Primary care physicians (PCPs) often face challenges in referring patients to the most appropriate specialists due to limited consultation time and incomplete knowledge of vast provider networks. Digital recommendation systems (RS) offer a potential solution but face three major hurdles:
- The Cold-Start Problem: Recommending doctors to patients the system has never seen before.
- Implicit Feedback: Visit history provides only binary data (visited or not) rather than explicit quality ratings.
- Privacy and Metadata: Strict regulations like GDPR limit the available patient metadata to basic demographics and location.
Methodology: XML and DECAF
The authors treat doctor recommendation as a Multilabel Classification problem where patients are instances and doctors are labels. Given the thousands of possible doctors, this is categorized as "Extreme" (XML). They utilize DECAF (Deep Extreme Classification with Label Features), a model that embeds both patient features and doctor labels into a unified vector space.
To evaluate the impact of different information types, the researchers defined five experimental scenarios (S-1 to S-5) based on combinations of patient (P) and doctor (D) metadata, including demographics, medical education, specialization history, and hospital proximity.
Key Results
The XML approach was compared against industry-standard recommendation models, including SVD, BiVAE, and LightFM, using a massive real-world dataset of 1 million patients and 5.5 million interactions from a European healthcare provider.
Performance Insights
- Superiority for Seen Patients: DECAF outperformed the strongest baseline (LightFM), improving PS-nDCG@3 (a measure of ranking quality) from 25.36 to 27.18 when all features were available.
- Cold-Start Robustness: The performance gap widened significantly for new patients. DECAF achieved a Recall@10 of 23.10, while LightFM trailed at 15.85. Traditional methods like SVD failed almost entirely in this scenario.
- Predictive Power of Proximity: For new patients without medical history, physical distance to hospitals (Scenario 4) proved to be the most effective predictor of specialist choice.
- Diversity in Recommendations: The XML framework proved more effective at recommending "tail" doctors—specialists who are less popular but may be the ideal fit—rather than defaulting to the most visited providers.
Conclusion
By reframing recommendation as an XML task, this research provides a robust framework for specialist referrals that excels even when patient data is sparse. The ability of the DECAF model to leverage both patient and doctor features in a shared space allows it to overcome traditional collaborative filtering limitations, particularly for the critical cold-start problem in healthcare.
Summary updated