TL;DR
This paper tackles the challenge of predicting periodontal disease (PD) by leveraging machine learning (ML) on a large dataset of electronic dental records (EDR). They developed an XGBoost model that not only predicts PD risk but also identifies novel risk factors, offering a more comprehensive approach than existing models.
What Problem Does This Paper Address?
Periodontal disease (PD) is a widespread issue affecting a significant portion of the population, leading to tooth loss and impacting quality of life. While preventive care is key, identifying high-risk patients early is crucial. Existing PD prediction models often suffer from suboptimal performance and may not offer new insights to clinicians, limiting their adoption. Furthermore, many models rely on traditional risk factors, potentially overlooking newer associations and the impact of social determinants of health. This study aims to bridge this gap by developing and testing a more accurate and insightful PD prediction model using machine learning and extensive electronic dental record data.
Methods
The core of this research lies in applying machine learning, specifically the XGBoost algorithm, to a large dataset of electronic dental records (EDR). To handle the nuances of EDR, they first developed computational approaches to automatically extract PD diagnoses and medical history information from clinical notes and charting data using natural language processing (NLP). Missing data was handled through imputation techniques, with categorical variables imputed using the most frequent value and continuous variables using the median.
The XGBoost model was trained using a "one-vs.-all" strategy to classify patients into three categories: healthy control, mild PD, and severe PD. This involved training three separate models: one to distinguish healthy individuals from all PD cases, another for mild PD versus others, and a third for severe PD versus others. The model's hyperparameters were optimized using a five-fold cross-validation strategy on the training set. Finally, the model was retrained on the entire training set and evaluated on a separate testing set. To understand the model's decision-making process, SHapley Additive exPlanations (SHAP) were used to interpret the contribution of each predictor.
Datasets
The study utilized electronic dental record (EDR) data from patients who received at least one comprehensive oral examination at the Temple University Kornberg School of Dentistry between January 1, 2017, and August 31, 2021. After data cleaning and exclusion of patients with excessive missing data, a final dataset of 18,553 unique patients was used. This dataset was split into an 80% training set (n=14,842) and a 20% testing set (n=3,711).
Key Insights and Ideas
- EDR as a Rich Source for PD Prediction: The study demonstrates the immense potential of leveraging large-scale EDR data, which captures a wide array of patient information beyond traditional risk factors, for developing sophisticated PD prediction models.
- XGBoost's Efficacy in Dental Health: The successful application of XGBoost, a powerful gradient boosting algorithm, highlights its suitability for complex medical prediction tasks, even in the dental domain.
- Beyond Traditional Risk Factors: The model identified novel associations and risk factors, including social determinants of health (e.g., distance from dental school), behavioral habits (e.g., recreational drug use), and systemic conditions, which were not always prioritized in older models.
- "One-vs.-All" for Granular Prediction: The use of a "one-vs.-all" strategy allowed for more nuanced predictions, distinguishing between different severity levels of PD, which is more clinically relevant than a simple healthy vs. diseased classification.
- SHAP for Clinical Interpretability: Employing SHAP values provides crucial interpretability, allowing clinicians to understand why a patient is flagged as high-risk, fostering trust and facilitating targeted interventions.
Key Results
The developed XGBoost model achieved an average Area Under the Curve (AUC) of 0.72 for distinguishing between healthy controls, mild PD, and severe PD cases. Specifically, the "one-vs.-rest" models showed an AUC of 0.69 for distinguishing healthy individuals from PD patients and 0.71 for distinguishing severe PD from others. The model identified 74 unique features from the EDR data. Notably, the study uncovered new associations, including patient anxiety, chewing problems, speaking trouble, teeth grinding, alcohol consumption, injury to teeth, and the presence of removable partial dentures, as well as medical conditions like osteoporosis, cancer, and various other systemic diseases.
Limitations and Future Work
The authors acknowledge several limitations:
- Single-Institute Dataset: The model was trained on data from a single institution, which may limit its generalizability to other populations.
- Limited ML Algorithms: Only XGBoost was tested; other advanced ML algorithms might yield superior performance.
- Self-Reported Data: Behavioral factors like smoking and alcohol use are self-reported, which can introduce bias.
- Simple Imputation: Basic imputation methods were used for missing data.
- Retrospective Nature: The study identified associations but cannot establish causation.
Future work suggested includes:
- External Validation: Evaluating the model's performance on external datasets to assess its generalizability.
- Longitudinal EDR Analysis: Analyzing longitudinal EDR data to understand the clinical progression of periodontitis.
- Usability Study: Conducting studies with dental clinicians to determine the model's practical usability and perception in clinical settings.
- Clinical Decision Support System: Developing a system that can be integrated into EDR to predict PD risk in real-time.
- Broader Data Integration: Incorporating additional data sources like geographic information systems, socioeconomic data, and criminal history to further enrich the prediction model.
Summary updated