Abstract
Colour differences between healthy and diseased tissue in the gastrointestinal tract are detected visually by clinicians during white light endoscopy (WLE); however, the earliest signs of disease are often just a slightly different shade of pink compared to healthy tissue. Here, we propose to target alternative colours for imaging to improve contrast using custom multispectral filter arrays (MSFAs) that could be deployed in an endoscopic chip-on-tip configuration. Using an open-source toolbox, Opti-MSFA, we examined the optimal design of MSFAs for early cancer detection in the gastrointestinal tract. The toolbox was first extended to use additional classification models (k-Nearest Neighbour, Support Vector Machine, and Spectral Angle Mapper). Using input spectral data from published clinical trials examining the oesophagus and colon, we optimised the design of MSFAs with 3 to 9 different bands. We examined the variation of the spectral and spatial classification accuracy as a function of number of bands. The MSFA designs have high classification accuracies, suggesting that future implementation in endoscopy hardware could potentially enable improved early detection of disease in the gastrointestinal tract during routine screening and surveillance. Optimal MSFA configurations can achieve similar classification accuracies as the full spectral data in an implementation that could be realised in far simpler hardware. The reduced number of spectral bands could enable future deployment of multispectral imaging in an endoscopic chip-on-tip configuration.
Introduction: The Challenge of Early Detection
In gastroenterology, early detection of neoplastic changes is critical for patient outcomes. However, the earliest signs of cancer in the esophagus or colon are often frustratingly subtle—clinicians describe them as just a "slightly different shade of pink" compared to healthy tissue. Standard White Light Endoscopy (WLE) relies on Red-Green-Blue (RGB) channels, which often lack the spectral resolution to distinguish these minute changes. While Hyperspectral Imaging (HSI) can distinguish these tissues by capturing hundreds of narrow color bands, the hardware is typically too bulky and slow for real-time clinical use. This research explores a "Goldilocks" solution: Multispectral Filter Arrays (MSFAs) that use a handful of optimized color bands to provide high accuracy in a compact, real-time "chip-on-tip" sensor.
Methodology: Optimizing the Spectral Mosaic
The researchers utilized an open-source Python toolbox, Opti-MSFA, to design custom sensors. The optimization process followed two main steps:
- Spectral Optimization: Gradient descent was used to identify the ideal center wavelengths and bandwidths (FWHM) for arrays containing 3 to 9 filters.
- Spatial Optimization: An exhaustive search determined the optimal mosaic arrangement of these filters on a camera sensor to minimize demosaicking errors.
The study extended the toolbox to include classification models such as k-Nearest Neighbour (kNN), Support Vector Machines (SVM), and Spectral Angle Mapper (SAM), alongside Linear Spectral Unmixing to track physiological markers like hemoglobin.
Clinical Datasets and Spectral Diversity
The study leveraged high-quality spectral data from two prior clinical trials to create synthetic hypercubes for testing.
As shown in the figure above, the Oesophageal Dataset included 715 spectra from 15 patients, categorized into healthy squamous tissue, non-dysplastic Barrett’s oesophagus (NDBE), and neoplasia. The Colon Dataset was significantly larger, containing 14,065 spectra from 13 patients, covering normal mucosa, polyps, and post-resection tissue.
Key Insights and Results
The optimization process revealed several critical findings for future hardware design:
- The Power of 4: Adding more than four bands yielded diminishing returns. A 3-band optimized MSFA achieved ~80% accuracy in the esophagus (vs. 84.8% for full HSI) and over 96% in the colon, matching full spectral performance.
- Biological Markers: Optimization consistently prioritized the 525–600 nm (green-yellow) region, which corresponds to hemoglobin signatures—a key indicator of angiogenesis in cancer.
- Intensity vs. Shape: In the esophagus, total reflected light intensity proved to be a vital diagnostic clue. Algorithms like SAM, which only consider spectral "shape," performed poorly compared to those that included intensity data.
- Spatial-Spectral Trade-off: The study demonstrated that for disease classification, a few high-quality spectral samples are often superior to many bands that might degrade spatial resolution or introduce noise.
Conclusion and Future Directions
This work provides a roadmap for developing "smart" endoscopes capable of seeing beyond the human eye's limits. By reducing the required spectral bands without sacrificing diagnostic accuracy, these designs enable the deployment of multispectral imaging in compact, real-time endoscopic configurations. Future work will focus on moving from synthetic simulations to physical prototypes, addressing real-world challenges like patient motion and specular reflections.
Summary updated