Abstract: Learning about diagnostic features and related clinical information from dental radiographs is important for dental research. However, the lack of expert-annotated data and convenient search tools poses challenges. Our primary objective is to design a search tool that uses a user's query for oral-related research. The proposed framework, Contrastive LAnguage Image REtrieval Search for dental research, Dental CLAIRES, utilizes periapical radiographs and associated clinical details such as periodontal diagnosis, demographic information to retrieve the best-matched images based on the text query. We applied a contrastive representation learning method to find images described by the user's text by maximizing the similarity score of positive pairs (true pairs) and minimizing the score of negative pairs (random pairs). Our model achieved a hit@3 ratio of 96% and a Mean Reciprocal Rank (MRR) of 0.82. We also designed a graphical user interface that allows researchers to verify the model's performance with interactions.
1. Introduction
In dental research, radiographs are the "gold standard" for diagnosis, yet they are often trapped in silos. While millions of scans are taken annually, finding a specific subset—for example, "Stage 3 periodontitis in a 60-year-old male"—is a manual, labor-intensive process. Existing repositories often lack a unified, searchable link between the visual content of a radiograph and the heterogeneous clinical data (demographics, medical history, diagnoses) stored in Electronic Health Records (EHR). Furthermore, manual expert annotation is expensive and slow. This paper introduces Dental CLAIRES, an automated, natural-language-driven retrieval system designed to bridge the gap between clinical text and dental imagery.
2. Methods
The proposed Dental CLAIRES (Contrastive LAnguage Image REtrieval Search) framework utilizes a contrastive representation learning approach to align multimodal data.
2.1 Dual Encoders and Shared Embedding Space
The system employs a dual-encoder architecture: DistilBERT for text encoding and ResNet-50 for image encoding. DistilBERT was selected for its efficiency, being 60% faster than standard BERT while maintaining 97% of its performance. Both text and image vectors are projected into a shared 256-dimensional space. The model is trained to maximize the cosine similarity between "true" pairs (a radiograph and its corresponding clinical description) and minimize it for "negative" pairs (randomly matched text and images).
2.2 Data Augmentation
To enhance model robustness, the authors implemented both image and text augmentations. Image augmentations included rotations and contrast shifts.
Text variation involved using Python scripts to replace clinical terms with synonyms (e.g., "maxilla" with "upper jaw") and creating caption variations with and without demographic data.
2.3 Inference
During inference, a user inputs a natural language query. The system encodes this query, compares it against a pre-indexed database of image embeddings using cosine similarity, and returns the top-K most similar matches.
3. Datasets
The study utilized a dataset of 687 periapical radiographs from 45 randomly selected periodontitis patients at the UTHealth School of Dentistry. Three independent experts (a professor, a clinician, and a resident) annotated the images for periodontal staging (Stages 1, 2, and 3) and anatomical position, with disagreements resolved via majority voting. Clinical metadata, including age, gender, and ethnicity, were extracted from EHR data to generate synthetic descriptive captions. The data was split into 80% training, 10% validation, and 10% testing, with care taken to ensure augmented versions of the same image remained within the same split to prevent data leakage.
4. Results and Discussion
The Dental CLAIRES model significantly outperformed baseline ablation models, achieving a Hit@3 ratio of 96% and a Mean Reciprocal Rank (MRR) of 0.82.
4.1 Performance Comparison
Ablation studies highlighted the importance of the multimodal approach:
- Dental CLAIRES: 0.72 Hit@1
- Text-Only Search: 0.66 Hit@1
- Image-Only Search: 0.16 Hit@1
The low performance of image-only search demonstrates that visual similarity alone is insufficient for clinical retrieval. Furthermore, the model remained robust across varying query complexities, maintaining an MRR of 0.86 for both "Low" (diagnosis only) and "Hard" (diagnosis + region + demographics) constraints.
4.2 Expert Variability
An analysis of Cohen’s Kappa scores revealed significant variability among human experts in periodontal staging, with agreement as low as 0.23 between some annotators. This underscores the need for "operator-independent" tools like Dental CLAIRES to provide a consistent baseline for research.
5. Conclusion
Dental CLAIRES demonstrates the potential of contrastive learning to unlock clinical insights from dental radiographs. While currently focused on periodontitis and a relatively small cohort, future work will involve scaling the dataset and expanding the scope to include other dental abnormalities (e.g., caries) and imaging modalities (e.g., panoramic X-rays). Integrating more complex EHR data, such as vitals and medication history, will further refine the system's retrieval capabilities.
Summary updated