Evaluation of synthetic training data for 3D intraoral reconstruction of cleft patients from single images

Lasse Lingens, Yoriko Lill, Prasad Nalabothu, Benito K. Benitez, Andreas A. Mueller, Markus Gross, Barbara SolenthalerInt. J. Comput. Assist. Radiol. Surg 2025 · OpenReview

Abstract

This study investigates the effectiveness of synthetic training data in predicting 2D landmarks for 3D intraoral reconstruction in cleft lip and palate patients. By leveraging high-fidelity synthetic data generated from real patient scans, we demonstrate a low-cost, non-invasive alternative to expensive 3D scanners and risky manual impressions, achieving results that approach clinical standards.

Introduction

Presurgical orthopedic (PSO) treatment for infants with cleft lip and palate requires custom-fitted plates to facilitate feeding and speech development. Creating these plates necessitates accurate 3D models of the infant's mouth. Traditional methods involve manual impressions, which pose choking risks, or expensive 3D intraoral scanners that are often inaccessible in low-resource settings. This research proposes a pipeline to reconstruct 3D models from a single smartphone image, bridging the gap in global cleft care accessibility.

Methodology

The proposed reconstruction pipeline consists of two primary stages: landmark prediction and statistical shape model (SSM) fitting.

Comparison of traditional intra-oral scanning versus the proposed smartphone-based 3D reconstruction pipeline.
  1. Landmark Prediction: A Convolutional Neural Network (CNN) is trained to detect 2D landmarks on intraoral images. To account for uncertainty, a negative-Gaussian log-likelihood loss function is employed to predict both landmark positions and confidence scores (variance).
  2. SSM Fitting: A Principal Component Analysis (PCA)-based shape model, trained on the general structure of cleft palates, is fitted to the predicted 2D landmarks to generate a 3D mesh.

The core innovation lies in the use of synthetic data. Due to the scarcity of annotated medical images, the team used 3D scans from 65 patients to render 9,000 synthetic images in Blender, simulating realistic lighting, textures, and camera angles.

Example of 2D landmark detection (sparse propagation) on an intraoral image of a cleft palate.

Results and Discussion

The study yielded several key insights regarding the use of synthetic training data:

  • Synthetic Data Utility: Synthetic data effectively substitutes for large real-world medical datasets, allowing for precise landmark annotation that would be impossible to label manually with the same precision.
  • Identity Diversity: Model performance is more dependent on the number of unique patient identities in the training set than the total image count. Adding variety in cleft shapes helps the model generalize better than adding more angles of the same patient.
  • Clinical Accuracy: The PCA model achieved a theoretical error of 0.34 mm, surpassing the 0.5 mm "gold standard" required for a well-fitting clinical plate.
  • Efficiency: The system performs landmark prediction in approximately 100ms, with full 3D fitting taking about 40 seconds, making it suitable for real-world clinical environments.

Despite these successes, a "domain gap" remains between synthetic and real-world images (RMSE of ~0.022 vs ~0.138), likely due to clinical variables like saliva and varying lighting conditions.

Conclusion

The use of synthetic data for training 3D reconstruction models shows significant promise for democratizing cleft palate care. Future work will focus on expanding the dataset of unique patient identities and integrating texture mapping to improve the realism and accuracy of the reconstructed models.

Summary updated

Abstract

This study investigates the effectiveness of synthetic training data in predicting 2D landmarks for 3D intraoral reconstruction in cleft lip and palate patients. We take inspiration from existing landmark prediction and 3D reconstruction techniques for faces and demonstrate their potential in medical applications.
BibTeX
@article{lingens2025evaluation,
  author = {Lasse Lingens and Yoriko Lill and Prasad Nalabothu and Benito K. Benitez and Andreas A. Mueller and Markus Gross and Barbara Solenthaler},
  title = {{Evaluation of synthetic training data for 3D intraoral reconstruction of cleft patients from single images}},
  journal = {Int. J. Comput. Assist. Radiol. Surg.},
  year = {2025},
  url = {https://openreview.net/forum?id=hS7SwCJ90K},
}
Figure