A Smartphone Application for Personalized Tooth Shade Determination

Tomoya Kusayanagi, Sota Maegawa, Shuya Terauchi, Wataru Hashimoto, Shohei KanedaDiagnostics 2023 · DOI

TL;DR

This paper introduces pTShaDe, a novel iPhone application that uses selfie-mode dental photographs to enable personalized tooth shade determination at home. By leveraging ambient light sensing and facial landmark detection, the app standardizes illumination and tooth appearance conditions, making it a practical tool for consumers to track the effectiveness of at-home teeth whitening products.

What Problem Does This Paper Address?

Evaluating the effectiveness of at-home teeth whitening products is challenging because existing methods for determining tooth shade are either subjective (visual methods using shade guides) or require expensive, specialized equipment and trained professionals (digital methods like colorimeters and spectrophotometers). This leaves consumers with limited options for accurately assessing their progress at home. The paper aims to bridge this gap by developing a user-friendly smartphone application that can provide a more objective and accessible way for individuals to monitor their tooth shade changes.

Methods

The core of the solution is the pTShaDe iPhone application, developed using Swift and Xcode. The app incorporates two key technologies to standardize dental photography for shade determination:

  • Ambient Light Sensing: An ambient light sensor is used to ensure consistent illumination. The app initializes the sensor and only activates the camera when the ambient light is near zero lux (dark conditions), mimicking a controlled environment. This is achieved by adjusting the iPhone's display brightness to adapt to ambient light levels.
  • Facial Landmark Detection (FLD): This AI technique is employed to maintain consistent tooth appearance. FLD identifies key facial features (eyes, nose, mouth, etc.) to provide geometric information. The app uses this to guide users in opening their mouth consistently and to overlay a semi-transparent image of the first captured photograph (with or without an outline) onto the live camera feed. This visual guide helps users align their mouth and teeth to match the initial reference image, ensuring a uniform "region of interest" (ROI) for color measurement across multiple captures.

The app captures photographs in selfie mode, allowing for easy self-operation. Tooth color is then analyzed using the CIELAB color space (L, a, b* values) from the captured incisor regions.

Datasets

The study did not utilize pre-existing datasets for training. Instead, it involved:

  • In Vitro Study: Photographs of four standard shade tabs (A1, A2, A3, A4) were captured using the app to evaluate the reliability of color measurement under controlled conditions.
  • In Vivo Study: Dental photographs were taken from seven participants. For the feasibility of in vivo tooth shade determination, one participant underwent pseudo-staining (coffee and grape juice) followed by gel whitening. Photographs were captured before and after each stage.

Key Insights and Ideas

  • Democratizing Dental Shade Assessment: The most significant insight is the successful development of a smartphone-based system that brings objective tooth shade determination out of the clinic and into the hands of consumers.
  • The Power of Standardization: The paper highlights how seemingly simple features like ambient light sensing and visual guides (semi-transparent images) are crucial for achieving reproducible results in smartphone photography for color measurement.
  • AI for Consistency: Facial landmark detection proves to be a surprisingly effective tool for ensuring consistent tooth appearance, a critical factor often overlooked in casual smartphone photography.
  • Relative vs. Absolute Measurement: The authors acknowledge that the method is a relative quantification, dependent on the smartphone model and environmental conditions. This pragmatic approach is key to its home-use feasibility.

Key Results

  • Tooth Appearance Uniformity: The coefficients of variation (CV) for the incisors' L, a, and b values across five sequential photographs were low: L (<0.0256), a (0.2748), and b (0.1053). This indicates good consistency in tooth appearance captured by the app.
  • Semi-transparent Image Effectiveness: While there were no significant differences in average distance or normalized area between using a semi-transparent image with or without an outline, some individual participants showed better results with one or the other, suggesting user preference might play a role.
  • In Vitro Reliability: The app demonstrated stability in distinguishing between shade tabs, with the smallest color difference (ΔEab) between tabs being 2.8.
  • In Vivo Feasibility: The app successfully monitored color changes after pseudo-staining and whitening. The smallest color difference (ΔEab) observed was 1.3, which the authors suggest as a potential detection limit for the app. This value is comparable to or better than previously reported thresholds for human perception of color differences.

Limitations and Future Work

  • Smartphone Dependency: The accuracy of the photographs is directly influenced by the smartphone's camera and flash performance, meaning results may not be directly comparable across different devices.
  • Environmental Constraints: The app requires dark conditions (0 lx) for optimal performance, necessitating users to create a controlled environment.
  • Multiple Photographs Required: To ensure accuracy, the app relies on multiple photographs, which can increase the time and potential for errors.
  • External Software Dependency: Currently, tooth color measurements require external software like Photoshop and MATLAB, which is not ideal for a seamless home-use experience.

Future Work: The authors suggest integrating a color measurement function directly into the app to eliminate the need for external software. They also propose adding more sophisticated criteria for tooth appearance, such as analyzing the shape of the hexagonal region to further refine mouth opening consistency.

Summary updated

Abstract

Tooth shade determination methods for evaluating the effectiveness of whitening products at home are limited. In this study, an iPhone app for personalized tooth shade determination was developed. While capturing dental photographs in selfie mode before and after whitening, the app can maintain consistent illumination and tooth appearance conditions that affect tooth color measurement. An ambient light sensor was used to standardize the illumination conditions. To maintain consistent tooth appearance conditions determined by appropriately opening the mouth, facial landmark detection, an artificial intelligence technique that estimates key face parts and outlines, was used. The effectiveness of the app in ensuring uniform tooth appearance was investigated through color measurements of the upper incisors of seven participants via photographs captured in succession. The coefficients of variation for incisors <i>L</i>*, <i>a</i>*, and <i>b</i>* were less than 0.0256 (95% CI, 0.0173-0.0338), 0.2748 (0.1596-0.3899), and 0.1053 (0.0078-0.2028), respectively. To examine the feasibility of the app for tooth shade determination, gel whitening after pseudo-staining by coffee and grape juice was performed. Consequently, whitening results were evaluated by monitoring the <i>∆E<sub>ab</sub></i> color difference values (1.3 unit minimum). Although tooth shade determination remains a relative quantification method, the proposed method can support evidence-based selection of whitening products.
BibTeX
@inproceedings{kusayanagi2023smartphone,
  author = {Tomoya Kusayanagi and Sota Maegawa and Shuya Terauchi and Wataru Hashimoto and Shohei Kaneda},
  title = {{A Smartphone Application for Personalized Tooth Shade Determination}},
  booktitle = {Diagnostics},
  year = {2023},
  doi = {10.3390/diagnostics13111969},
}
Figure