Dual-modality, deep-learning-enabled endomicroscope with large field-of-view and depth-of-field for real-time in vivo imaging of epithelial hallmarks of cancer

Huayu Hou, Jimin Wu, Jinyun Liu, Vivek Boominathan, Argaja Deepak Shende, Karthik Goli, Jennifer Carns, Richard A. Schwarz, Ann M. Gillenwater, Preetha Ramalingam, Mila P. Salcedo, Kathleen M. Schmeler, Tomasz Tkaczyk, Jacob T. Robinson, Ashok Veeraraghavan, Rebecca Richards‐KortumbioRxiv 2026 · DOI

TL;DR

Researchers have developed "PrecisionView," a compact, AI-powered endomicroscope that overcomes the traditional trade-offs in in-vivo microscopy. By combining a deep-learning optimized phase mask with real-time reconstruction, it achieves a significantly larger field-of-view and depth-of-field, enabling high-resolution, dual-modality imaging of cellular and vascular structures for improved early cancer detection at the point-of-care.

What Problem Does This Paper Address?

Early detection of epithelial cancers is crucial for improving patient outcomes, but current diagnostic methods like histopathology are invasive and can suffer from sampling errors. In-vivo microscopy (IVM) offers a promising alternative for real-time imaging of precancerous changes, but conventional IVM systems are limited by fundamental optical trade-offs: a small field-of-view (FOV) makes it difficult to assess large, heterogeneous lesions, and a shallow depth-of-field (DOF) requires constant refocusing, hindering efficient examination. These limitations restrict the ability to capture comprehensive cellular and vascular information needed for accurate diagnosis, especially in challenging, irregular tissue surfaces.

Methods

The core innovation of PrecisionView lies in its "end-to-end deep learning optimization framework." This approach co-optimizes both the optical hardware (specifically, a custom-designed phase mask) and the image reconstruction software (a deep neural network called PrecisionNet).

  • Phase Mask: A specially designed phase mask is integrated into the optical path. This mask modulates the light wavefront in a way that, when combined with the deep learning reconstruction, generates spatially and depth-invariant point-spread functions (PSFs). This is key to achieving the extended DOF.
  • Deep Learning Reconstruction (PrecisionNet): A modified U-Net architecture is employed to process the blurry images captured by the system. This network is trained to deconvolve the optical aberrations and defocus introduced by the system, effectively reconstructing sharp, in-focus images.
  • Dual-Modality Imaging: The system supports both fluorescence imaging (for cell nuclei) and reflectance imaging (for microvasculature), allowing for the simultaneous assessment of two critical hallmarks of cancer.
  • Real-time Processing: The reconstruction algorithm is designed for real-time performance, enabling immediate visualization of images during scanning.

Datasets

The paper describes the creation of a large and diverse training dataset for the deep learning model. This dataset was generated using a forward model simulator and included:

  • Open-source microscopy datasets.
  • Histopathology images of healthy and cancerous human tissues from The Cancer Genome Atlas (TCGA).
  • Widefield microscopy images of vascular features.
  • Microscopy images from fluorescence and reflectance modalities.

The authors note that collecting large-scale, paired datasets for lensless imaging systems is challenging, and their approach of using a simulator and diverse real-world data was crucial for training.

Key Insights and Ideas

  • Hardware-Software Co-optimization: The paper strongly emphasizes that significant improvements in imaging capabilities (FOV, DOF) are achieved not just by better optics or better algorithms, but by jointly optimizing them using deep learning. This "end-to-end" approach unlocks performance beyond what either component could achieve alone.
  • Depth-Invariant PSFs: The phase mask design is crucial for creating PSFs that are less sensitive to depth variations. This is a departure from traditional optics where PSF changes dramatically with focus.
  • Real-time Blind Deblurring: The system performs blind deblurring (recovering sharp images from unknown blur) in real-time, which is a significant computational feat enabling immediate clinical feedback.
  • Dual-Modality for Comprehensive Assessment: The ability to simultaneously image cell nuclei and microvasculature is highlighted as a key advantage, providing a more complete picture of tissue health and pathology than single-modality imaging.

Key Results

  • Extended FOV and DOF: PrecisionView achieves a 5.2 mm × 3.9 mm FOV (approx. 5x increase) and a 500 µm DOF (approx. 8x increase) compared to conventional IVM systems with similar resolution (4 µm).
  • Resolution: Maintains a lateral resolution of approximately 4 µm across the entire FOV and DOF.
  • Imaging Speed: Real-time reconstruction at up to 15 FPS for dual-modality imaging, and up to 30 FPS for reflectance imaging.
  • Large-Scale Mapping: Capable of generating co-registered maps of cellular and vascular structures over large tissue areas (1-3 cm² in vivo, > 2.8 cm² ex vivo) through manual scanning and image stitching.
  • Clinical Validation: Demonstrated ability to visualize distinct morphological features in healthy oral mucosa, porcine tongue, human breast tissue, and crucially, to differentiate precancerous cervical lesions from normal tissue.

Limitations and Future Work

  • Manual Scanning: While the system enables large-scale mapping, it currently relies on manual scanning and image stitching. Future work could explore automated scanning for increased efficiency and consistency.
  • Computational Power: While designed for minimal computational power, real-time reconstruction speed is dependent on hardware. Further optimization of the network or hardware could improve frame rates.
  • Low-Resource Settings: The authors suggest that the compact size, affordability ($3,000 cost of goods), and potential for automated interpretation could make this technology particularly valuable in low-resource settings.
  • Further Clinical Studies: The paper calls for larger in vivo clinical studies to rigorously assess diagnostic performance and develop computer-aided diagnostic algorithms for automated interpretation.
  • Adaptability: The modular design allows for adaptation to other contrast agents (e.g., methylene blue) for imaging different cellular features.

Summary updated

Abstract

In vivo microscopy (IVM) has shown great promise to improve early detection of epithelial precancer, but it suffers from fundamental trade-offs that limit the resolution, field-of-view (FOV) and depth-of-field (DOF). Here, we present PrecisionView, a compact, deep-learning-enabled endomicroscope that breaks these constrains and achieves 20 mm <sup>2</sup> FOV and 500 µm DOF with 4 µm resolution, representing approximately 5× increase in FOV and 8× larger DOF compared to conventional IVM with similar resolution. PrecisionView integrates a deep-learning optimized phase mask and real-time reconstruction, enabling rapid in vivo assessment of two key hallmarks of cancer: epithelial cell nuclear morphology and subsurface microvasculature through fluorescence and reflectance imaging. By imaging oral cavity of healthy volunteers and cervical specimens with precancerous lesions, PrecisionView generates large-scale (1-3 cm <sup>2</sup> ) co-registered maps of cellular and vascular structures, revealing distinct microscopic patterns associated with anatomic structures and precancerous lesions. Our results suggest the potential of this computational endomicroscope to address the unmet need for early cancer detection at the point-of-care.
BibTeX
@misc{hou2026dual,
  author = {Huayu Hou and Jimin Wu and Jinyun Liu and Vivek Boominathan and Argaja Deepak Shende and Karthik Goli and Jennifer Carns and Richard A. Schwarz and Ann M. Gillenwater and Preetha Ramalingam and Mila P. Salcedo and Kathleen M. Schmeler and Tomasz Tkaczyk and Jacob T. Robinson and Ashok Veeraraghavan and Rebecca Richards‐Kortum},
  title = {{Dual-modality, deep-learning-enabled endomicroscope with large field-of-view and depth-of-field for real-time in vivo imaging of epithelial hallmarks of cancer}},
  year = {2026},
  doi = {10.64898/2026.01.14.699592},
  howpublished = {bioRxiv},
}
Figure