A machine learning framework for automated diagnosis and computer-assisted planning in plastic and reconstructive surgery

Paul G.m. Knoops, Athanasios Papaioannou, Alessandro Borghi, Richard W.f. Breakey, Alexander T. Wilson, Owase Jeelani, Stefanos Zafeiriou, Derek M. Steinbacher, Bonnie L. Padwa, David Dunaway, Silvia SchievanoScientific 2019 · DOI

Abstract

Current computational tools for planning and simulation in plastic and reconstructive surgery lack sufficient precision and are time-consuming, resulting in limited clinical adoption. Here, we present the first large-scale clinical 3D morphable model (3DMM), a machine-learning-based framework involving supervised learning for diagnostics, risk stratification, and treatment simulation. The model, trained and validated with 4,261 faces of healthy volunteers and orthognathic surgery patients, diagnoses patients with 95.5% sensitivity and 95.2% specificity, and simulates surgical outcomes with a mean accuracy of 1.1 ± 0.3 mm. This framework demonstrates the potential for fully-automated diagnosis and patient-specific treatment planning from a single 3D scan.

Introduction

In plastic and reconstructive surgery—specifically orthognathic (jaw) surgery—precision is paramount. While computer-assisted surgical planning (CASP) systems exist, they are often manual and complex, requiring surgeons to iteratively test bone movements to visualize final outcomes. This time-consuming process limits the use of CASP in routine consultations. There is a clear need for automated, high-precision tools that can objectively diagnose facial deformities and predict surgical results using non-invasive 3D surface scans.

Methodology

The researchers developed a pipeline centered on 3D Morphable Models (3DMMs), which represent facial shape as a linear combination of principal components. The pipeline involves four main blocks:

  1. Automatic Annotation: Using an Active Appearance Model (AAM) to identify landmarks on 3D meshes.
  2. Alignment: Removing variations in rotation, scale, and translation via Procrustes analysis.
  3. Dense Correspondence: Ensuring consistent anatomical vertex locations across scans using non-rigid Iterative Closest Point (NICP) algorithms.
  4. Statistical Analysis: Applying Principal Component Analysis (PCA) to identify the primary modes of facial variation.

For diagnosis, a Support Vector Machine (SVM) was utilized to classify faces. For surgical simulation, regression techniques (Linear, Ridge, LARS, and LASSO) were tested to map preoperative shapes to predicted postoperative outcomes.

Model Performance and the 'Bespoke' Advantage

A critical insight from this study is the advantage of "bespoke" models—those trained specifically on clinical patient data—over general population models like the Large Scale Facial Model (LSFM). While general models capture broad population variance, they lack the specific morphological details found in surgical cases.

Comparison of variance retention and generalisation error across LSFM, Global, and Bespoke models.

As illustrated in the figure above, the bespoke preoperative model retains more variance with fewer principal components (Panel a) and achieves significantly lower generalisation error (Panel b) compared to the LSFM. This demonstrates that clinical shape variation is a distinct biomarker requiring specialized modeling.

Clinical Outcomes and Diagnostic Accuracy

The framework demonstrated exceptional diagnostic performance, achieving 95.5% sensitivity and 95.2% specificity. Beyond classification, the model's precision in representing unseen faces (specificity error) is a key metric for its reliability in a clinical setting.

Specificity error of the bespoke preoperative model as a function of principal components.

The specificity error for the bespoke model (shown above) stabilizes at approximately 0.40 mm, indicating high representational accuracy. Furthermore, the model predicted postoperative outcomes with a mean error of 1.1 ± 0.3 mm, which is on par with or superior to current manual commercial software.

Discussion and Limitations

This study establishes face shape as a quantitative biomarker, moving away from subjective aesthetic assessments. By utilizing Ridge Regression and Least-Angle Regression, the model provides patient-specific predictions that avoid the "mean trap" of simply predicting an average face.

However, limitations remain. The current model relies on surface scans and lacks volumetric bone data; it predicts the final appearance but not the specific bone movements required. Additionally, the model had a 12.5% false-negative rate, likely because surgery is often driven by functional issues (e.g., breathing or chewing) that surface shape alone cannot detect. Future work will focus on multimodal integration, combining 3D shape with electronic medical records and genetic data.

Conclusion

The developed 3DMM framework offers a path toward fully automated, objective, and high-precision surgical planning. By significantly reducing the time required for preoperative workup, this technology can improve clinical decision-making and enhance patient understanding of surgical outcomes.

Summary updated

Abstract

Current computational tools for planning and simulation in plastic and reconstructive surgery lack sufficient precision and are time-consuming, thus resulting in limited adoption. Although computer-assisted surgical planning systems help to improve clinical outcomes, shorten operation time and reduce cost, they are often too complex and require extensive manual input, which ultimately limits their use in doctor-patient communication and clinical decision making. Here, we present the first large-scale clinical 3D morphable model, a machine-learning-based framework involving supervised learning for diagnostics, risk stratification, and treatment simulation. The model, trained and validated with 4,261 faces of healthy volunteers and orthognathic (jaw) surgery patients, diagnoses patients with 95.5% sensitivity and 95.2% specificity, and simulates surgical outcomes with a mean accuracy of 1.1 ± 0.3 mm. We demonstrate how this model could fully-automatically aid diagnosis and provide patient-specific treatment plans from a 3D scan alone, to help efficient clinical decision making and improve clinical understanding of face shape as a marker for primary and secondary surgery.
BibTeX
@inproceedings{knoops2019machine,
  author = {Paul G.M. Knoops and Athanasios Papaioannou and Alessandro Borghi and Richard W.F. Breakey and Alexander T. Wilson and Owase Jeelani and Stefanos Zafeiriou and Derek M. Steinbacher and Bonnie L. Padwa and David Dunaway and Silvia Schievano},
  title = {{A machine learning framework for automated diagnosis and computer-assisted planning in plastic and reconstructive surgery}},
  booktitle = {Scientific},
  year = {2019},
  doi = {10.1038/s41598-019-49506-1},
}
Figure