DENTEX: An Abnormal Tooth Detection with Dental Enumeration and Diagnosis Benchmark for Panoramic X-rays

Ibrahim Ethem Hamamci, Sezgin Er, Enis Simsar, Atif Emre Yüksel, Sadullah Gultekin, Serife Damla Ozdemir, Kaiyuan Yang, Hongwei Bran Li, Sarthak Pati, Bernd Stadlinger, Albert Mehl, Mustafa Gundogar, Bjoern H. Menze

Abstract

Panoramic X-rays are frequently used in dentistry for treatment planning, but their interpretation can be both time-consuming and prone to error. Artificial intelligence (AI) has the potential to aid in the analysis of these X-rays, thereby improving the accuracy of dental diagnoses and treatment plans. Nevertheless, designing automated algorithms for this purpose poses significant challenges, mainly due to the scarcity of annotated data and variations in anatomical structure. To address these issues, the Dental Enumeration and Diagnosis on Panoramic X-rays Challenge (DENTEX) has been organized in association with the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) in 2023. This challenge aims to promote the development of algorithms for multi-label detection of abnormal teeth, using three types of hierarchically annotated data: partially annotated quadrant data, partially annotated quadrant-enumeration data, and fully annotated quadrant-enumeration-diagnosis data, inclusive of four different diagnoses. In this paper, we present the results of evaluating participant algorithms on the fully annotated data, additionally investigating performance variation for quadrant, enumeration, and diagnosis labels in the detection of abnormal teeth. The provision of this annotated dataset, alongside the results of this challenge, may lay the groundwork for the creation of AI-powered tools that can offer more precise and efficient diagnosis and treatment planning in the field of dentistry.

Introduction

In clinical dentistry, panoramic X-rays (orthopantomograms) are essential for treatment planning. However, manual interpretation is time-consuming and prone to human error. While AI has shown promise in medical imaging, dental AI faces a significant "data gap." Most existing datasets are small, poorly annotated, or focus on descriptive effects rather than actionable clinical diagnoses.

Furthermore, dental images present unique geometric challenges. Teeth are small, tightly packed, and vary significantly between patients. Automated systems must not only locate a tooth but also correctly identify its "enumeration" (specific position/number in the mouth) and detect subtle pathologies like periapical lesions, which often exhibit low contrast against the bone.

Dataset and Hierarchical Annotation

The DENTEX dataset was introduced to address these challenges through a hierarchical structure that encourages learning from partial labels. This structure allows models to progressively learn more complex tasks, from broad anatomical regions to specific tooth IDs and finally to clinical pathologies.

  • Quadrant-only: 693 X-rays labeled for quadrants.
  • Quadrant-Enumeration: 634 X-rays labeled for quadrants and tooth enumeration using the FDI numbering system.
  • Full Annotation: 1,005 X-rays fully annotated for quadrant, enumeration, and four clinical diagnoses: Caries, Deep caries, Periapical lesion, and Impacted teeth.
  • Unlabeled Data: 1,571 images provided for self-supervised pre-training.
The DENTEX hierarchical annotation scheme showing (a) quadrant detection, (b) tooth enumeration, and (c) clinical diagnosis detection.

Methodology

The top-performing strategies from the DENTEX challenge generally utilized one of three architectural approaches:

  1. Segmentation-Guided Pipelines: The winning team (He L.) employed a "segmentation-first" approach. They used a U-Net to create precise pixel-level maps of teeth before passing them to a detection model (DINO). This effectively "de-cluttered" the scene for the detector, preventing instance merging of adjacent teeth.
  2. Specialized Single-Stage Detectors: Team Mei S. modified YOLOv8 into "YOLOrtho." Key innovations included coordinate convolutions, which provide the network with an explicit sense of absolute spatial position—crucial for identifying quadrants—and the Hungarian algorithm to ensure unique tooth ID assignment.
  3. Iterative Refinement (Diffusion Models): The baseline model (HierarchicalDet) and several top entries leveraged diffusion-based detection. Unlike standard detectors that make a single pass, these models iteratively refine "noisy" boxes into precise detections, which proved particularly effective for identifying subtle textural signals in periapical lesions.

Results and Performance Analysis

The evaluation of participant algorithms across 12 metrics provided a comprehensive view of the current state of dental AI:

  • Top Overall Performer: Team He L. achieved the best overall performance with a mean rank of 2.33.
  • Recall vs. Precision: Team Choi K. focused on a high-recall strategy (achieving the highest Average Recall), which is ideal for screening tools where missing a pathology is the primary risk.
  • Diagnosis Accuracy: On the most difficult task—diagnosis—the diffusion-based baseline (HierarchicalDet) outperformed many complex ensembles, ranking 1st in Average Precision (AP) for disease detection.
Performance metrics (Average Recall and Average Precision) for participant algorithms across the three detection tasks: (a) Quadrant, (b) Enumeration, and (c) Diagnosis.

Key insights from the results suggest that segmentation is a powerful prior and that spatial awareness is vital for dental imaging. Adding coordinate convolutions provided a necessary "inductive bias," allowing models to learn that specific quadrants are always located in relative spatial areas.

Discussion and Future Work

Despite the progress shown in DENTEX, several hurdles remain for clinical deployment. Panoramic X-rays provide a broad overview, but definitive diagnosis for caries often requires supplemental intraoral (bitewing) X-rays. Future benchmarks should explore multi-modality AI. Furthermore, dental diagnosis involves significant inter-observer disagreement; future work could incorporate "multi-rater" labels to capture this clinical uncertainty.

Finally, current metrics like Average Precision treat all errors equally. In practice, swapping the numbers of two adjacent teeth is a minor error, while missing an impacted molar entirely is a significant clinical failure. New, clinically-weighted metrics are needed to better align AI performance with dental practice. The evaluation code and datasets are publicly available to serve as a permanent benchmark for the community.

Summary updated

Abstract

Panoramic X-rays are frequently used in dentistry for treatment planning, but their interpretation can be both time-consuming and prone to error. Artificial intelligence (AI) has the potential to aid in the analysis of these X-rays, thereby improving the accuracy of dental diagnoses and treatment plans. Nevertheless, designing automated algorithms for this purpose poses significant challenges, mainly due to the scarcity of annotated data and variations in anatomical structure. To address these issues, the Dental Enumeration and Diagnosis on Panoramic X-rays Challenge (DENTEX) has been organized in association with the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) in 2023. This challenge aims to promote the development of algorithms for multi-label detection of abnormal teeth, using three types of hierarchically annotated data: partially annotated quadrant data, partially annotated quadrant-enumeration data, and fully annotated quadrant-enumeration-diagnosis data, inclusive of four different diagnoses. In this paper, we present the results of evaluating participant algorithms on the fully annotated data, additionally investigating performance variation for quadrant, enumeration, and diagnosis labels in the detection of abnormal teeth. The provision of this annotated dataset, alongside the results of this challenge, may lay the groundwork for the creation of AI-powered tools that can offer more precise and efficient diagnosis and treatment planning in the field of dentistry. The evaluation code and datasets can be accessed at https://github.com/ibrahimethemhamamci/DENTEX
BibTeX
@misc{hamamci2023dentex,
  author = {Ibrahim Ethem Hamamci and Sezgin Er and Enis Simsar and Atif Emre Yüksel and Sadullah Gultekin and Serife Damla Ozdemir and Kaiyuan Yang and Hongwei Bran Li and Sarthak Pati and Bernd Stadlinger and Albert Mehl and Mustafa Gundogar and Bjoern H. Menze},
  title = {{DENTEX: An Abnormal Tooth Detection with Dental Enumeration and Diagnosis Benchmark for Panoramic X-rays}},
  year = {2023},
  eprint = {2305.19112},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.19112},
}
Figure