Low-cost, sub-micron resolution, wide-field computational microscopy using opensource hardware

Tomas Aidukas, Regina Eckert, Andrew R. Harvey, Laura Waller, Pavan Chandra Konda

TL;DR

This paper presents a low-cost, wide-field, sub-micron resolution computational microscope built using off-the-shelf components like a Raspberry Pi and a mobile phone camera. By combining Fourier ptychography with novel image reconstruction techniques tailored for Bayer color sensors and aberrated lenses, they achieve 25-megapixel images, significantly democratizing high-performance microscopy for resource-limited settings.

What Problem Does This Paper Address?

High-resolution microscopy is crucial for biological and medical applications, but conventional systems are often expensive, bulky, and have limited fields of view (FOV). While low-cost alternatives exist, they typically compromise on either resolution or FOV, failing to provide the "space-bandwidth product" (SBP) needed for efficient screening of large areas for sub-cellular features. This paper tackles the challenge of achieving both high resolution and wide FOV in a cost-effective, portable microscope, making advanced imaging accessible for applications ranging from drug discovery to global health screening.

Methods

The core of the system is a Fourier ptychography (FP) microscope. FP works by computationally reconstructing a high-resolution image from a series of low-resolution images captured with different illumination angles. This allows for the stitching together of frequency information that would otherwise be outside the optical system's native passband, effectively increasing the SBP.

Key innovations in this work include:

  • Open-source hardware: Utilizing a Raspberry Pi single-board computer and a standard mobile phone camera module (with a 3D-printed opto-mechanical structure) drastically reduces cost.
  • Bayer color sensor adaptation: Mobile phone cameras use Bayer color filters, which sparsely sample color information. The authors developed new reconstruction algorithms to handle this sparse sampling, overcoming the limitations of traditional demosaicing. They compared two approaches: sparsely-sampled reconstruction (SSR) and demosaiced reconstruction (DR), finding DR to be more robust.
  • Aberration correction: The use of inexpensive, re-used camera lenses introduces significant aberrations, especially towards the edges of the FOV. The paper details algorithmic self-calibration techniques to compensate for these optical imperfections and LED misalignment.
  • Computational reconstruction: The FP reconstruction algorithm is adapted to handle the Bayer sensor's sparsity and the system's aberrations. This involves iterative phase retrieval and pupil recovery.

Datasets

The paper uses simulated data for evaluating reconstruction algorithms and experimental data from a USAF resolution test chart and lung carcinoma tissue samples for performance assessment. No new benchmark datasets are introduced.

Key Insights and Ideas

  • Democratizing high-resolution imaging: The most significant insight is that high-performance microscopy (sub-micron resolution, wide FOV) is achievable with consumer-grade electronics and open-source hardware, drastically lowering the barrier to entry.
  • Bayer sensor challenge is surmountable: The sparse sampling inherent in Bayer color filters, a common feature in low-cost cameras, was a major hurdle. The development of tailored reconstruction algorithms demonstrates that this is not an insurmountable obstacle for advanced computational imaging.
  • Aberration correction is key for low-cost optics: The success of the system hinges on sophisticated computational methods to correct for the severe aberrations introduced by inexpensive lenses. This highlights the power of computational imaging to overcome optical limitations.
  • Self-calibration for robustness: The system's ability to self-calibrate LED positions and correct for mechanical misalignments makes it practical for portable and field use, where stability might be compromised.

Key Results

  • 25-megapixel images: The system reconstructs images with a 25-megapixel SBP from 4-megapixel raw sensor data.
  • Sub-micron resolution: Achieved a resolution of 780 nm (corresponding to a synthetic NA of 0.55), a significant improvement over previous low-cost microscopy efforts.
  • Wide field of view: Demonstrated a 4 mm² FOV, a 5-fold increase in area compared to a previous 3D-printed FP microscope.
  • Cost reduction: The component cost is approximately $150, a fraction of traditional high-performance microscopes.
  • Resolution improvement: A three-fold resolution improvement was observed on a USAF test chart, from 2.8 µm to 780 nm.
  • Aberration correction impact: Computational correction of aberrations increased the usable FOV by a factor of four.

Limitations and Future Work

  • Acquisition time: While improved, image acquisition can still take up to 45 minutes for a full dataset (256 images). The authors suggest LED multiplexing and multi-camera approaches to reduce this significantly, potentially to under a minute.
  • Computational intensity: Reconstruction is computationally intensive, requiring transfer to a PC. While on-board reconstruction is possible with optimization, it's not yet the primary mode.
  • Chromatic aberrations: While addressed, severe chromatic aberrations from low-cost lenses can still pose challenges, requiring careful calibration for each color channel.
  • Future work: The authors suggest further reducing acquisition time, exploring on-board reconstruction, and investigating neural network approaches for faster image recovery. They also highlight the potential for this technology in various applications, including point-of-care diagnostics and educational tools.

Summary updated

Abstract

The revolution in low-cost consumer photography and computation provides fertile opportunity for a disruptive reduction in the cost of biomedical imaging. Conventional approaches to low-cost microscopy are fundamentally restricted, however, to modest field of view (FOV) and/or resolution. We report a low-cost microscopy technique, implemented with a Raspberry Pi single-board computer and color camera combined with Fourier ptychography (FP), to computationally construct 25-megapixel images with sub-micron resolution. New image-construction techniques were developed to enable the use of the low-cost Bayer color sensor, to compensate for the highly aberrated re-used camera lens and to compensate for misalignments associated with the 3D-printed microscope structure. This high ratio of performance to cost is of particular interest to high-throughput microscopy applications, ranging from drug discovery and digital pathology to health screening in low-income countries. 3D models and assembly instructions of our microscope are made available for open source use.
BibTeX
@inproceedings{aidukas2019low,
  author = {Tomas Aidukas and Regina Eckert and Andrew R. Harvey and Laura Waller and Pavan Chandra Konda},
  title = {{Low-cost, sub-micron resolution, wide-field computational microscopy using opensource hardware}},
  booktitle = {Scientific},
  year = {2019},
  doi = {10.1038/s41598-019-43845-9},
}
Figure