Drug classification with a spectral barcode obtained with a smartphone Raman spectrometer

Un Jeong Kim, Su‐Yeon Lee, Hyochul Kim, Yeongeun Roh, Seungju Han, Hojung Kim, Yeonsang Park, Seokin Kim, Myung Jin Chung, Hyungbin Son, Hyuck ChooNature Communications 2023 · DOI

TL;DR

Researchers have developed a smartphone-based Raman spectrometer that uses a custom 2D filter array to generate unique "spectral barcodes" for medications. By pairing this hardware with a convolutional neural network (CNN), the system identifies major drug components with 99% accuracy and can even distinguish between different brand names of the same drug.

What Problem Does This Paper Address?

The rise of online pharmacies and complex global supply chains has led to a surge in counterfeit and substandard drugs, posing a significant threat to public health. While smartphone apps exist to identify pills based on visual cues like color, shape, and imprinted codes, these are easily mimicked by counterfeiters.

True chemical verification usually requires Raman spectroscopy—a "fingerprinting" technique that identifies molecules based on how they scatter laser light. However, traditional Raman spectrometers are bulky, expensive, and difficult to miniaturize. Standard attempts to put spectrometers in phones often use gratings to disperse light, which struggle to maintain the high resolution and signal-to-noise ratio (SNR) needed for complex chemical analysis in a tiny form factor.

Methods

The authors took a hardware-software co-design approach to bypass the limitations of traditional miniaturized optics:

  1. The Hardware: They attached a compact external module containing a 785 nm laser diode to a Samsung Galaxy Note 9.
  2. The "Spectral Barcode": Instead of using a grating to create a standard 1D spectrum, they integrated a 2D periodic array of 120 distinct bandpass filters (covering the 830–910 nm range) directly over the phone's CMOS image sensor. This creates a 2D intensity map where each "pixel" represents a specific wavelength.
  3. The Symbology: They defined this 2D map as a "spectral barcode." This shift is crucial: rather than trying to reconstruct a perfect scientific spectrum, they treated the data as an image for a computer vision model to interpret.
  4. The AI: They utilized a dual-track CNN approach. A simplified ResNet architecture processed the spectral barcodes to identify chemical components, while a VGGNet processed standard RGB photos of the pills to identify physical characteristics (color and shape).

Datasets

The study utilized a custom dataset of 54 commonly prescribed drugs (covering conditions like diabetes, hypertension, and hyperlipidemia) and nutritional supplements.

  • Training/Testing Data: 170–223 Raman images and ~200 RGB images per drug.
  • Generalization Test: Four "unknown" drugs (not included in the training set but containing known active ingredients) were used to see if the model could identify components it had "learned" from other brands.

Key Insights and Ideas

  • Image over Line Graphs: The "aha moment" here is treating spectral data as a 2D image (a barcode) rather than a 1D signal. This allows the system to leverage the massive advancements in image-recognition CNNs, which are remarkably robust at finding patterns even in noisy or low-resolution data.
  • Fluorescence as a Feature, Not a Bug: In traditional spectroscopy, fluorescence background is often seen as noise to be removed. Here, the authors found that the specific fluorescence "hump" caused by brand-specific additives and coatings actually helped the CNN distinguish between different brands of the same generic drug.
  • Information Density: The 120-channel filter array provides roughly 1200 bits of chemical information. This is comparable to a standard 2D QR code but represents the intrinsic molecular properties of the substance itself.

Key Results

  • Component Accuracy: The system achieved 99.0% accuracy in classifying 11 major chemical components across the 54-drug library.
  • Unknown Drug Identification: For drugs not in the training database, the CNN correctly identified the major chemical component with 99.8% accuracy.
  • Brand Distinction: By fusing the spectral barcode data with standard RGB image data (color/shape), the system could identify the specific brand name with 83.2% accuracy.
  • Speed: The entire process, from light collection to result, takes only a few seconds, making it viable for real-world consumer use.

Limitations and Future Work

While impressive, the system currently relies on an external laser module, which limits its "out-of-the-box" portability for the average consumer. The authors also noted that thick pill coatings can sometimes obscure the Raman signal of the inner core; they suggest that future versions could use Spatially Offset Raman Spectroscopy (SORS) to "see" through these layers.

Finally, the current spectral range is limited to 830–910 nm. Expanding this range by adding more filters would allow the system to detect a wider variety of chemical bonds, further increasing its diagnostic power.

Summary updated

Abstract

Measuring, recording and analyzing spectral information of materials as its unique finger print using a ubiquitous smartphone has been desired by scientists and consumers. We demonstrated it as drug classification by chemical components with smartphone Raman spectrometer. The Raman spectrometer is based on the CMOS image sensor of the smartphone with a periodic array of band pass filters, capturing 2D Raman spectral intensity map, newly defined as spectral barcode in this work. Here we show 11 major components of drugs are classified with high accuracy, 99.0%, with the aid of convolutional neural network (CNN). The beneficial of spectral barcodes is that even brand name of drug is distinguishable and major component of unknown drugs can be identified. Combining spectral barcode with information obtained by red, green and blue (RGB) imaging system or applying image recognition techniques, this inherent property based labeling system will facilitate fundamental research and business opportunities.
BibTeX
@inproceedings{kim2023drug,
  author = {Un Jeong Kim and Su‐Yeon Lee and Hyochul Kim and Yeongeun Roh and Seungju Han and Hojung Kim and Yeonsang Park and Seokin Kim and Myung Jin Chung and Hyungbin Son and Hyuck Choo},
  title = {{Drug classification with a spectral barcode obtained with a smartphone Raman spectrometer}},
  booktitle = {Nature Communications},
  year = {2023},
  doi = {10.1038/s41467-023-40925-3},
}
Figure