Deep Depth-of-Field for Microscopy On-the-Go
Lists
Only you see this
Abstract
The datasets accompany the paper: “Deep Depth-of-Field for Microscopy On-the-Go”, by Judith Fischer, Aviad Avni, Naama Pearl, Amit Peleg, Tali Treibitz, the Hatter Department of Marine Technologies , Leon H. Charney School of Marine Sciences, University of Haifa, Israel. Paper abstract: A small imaging volume is a very limiting factor in microscopy, especially when aiming at imaging microscopic organisms in their dynamic natural environment, e.g., outdoors or underwater. Here we propose a method that is purely computational, does not require special optics, and results in a Deep-Depth of Field (D-DOF) -- up to X50 larger that the original DOF, with only a small penalty in acquisition time and quality. The scene is acquired using one-shot imaging, where focal scanning is performed with the shutter kept open. The acquired single one-shot image is blurry due to the superimposed focal planes. Previously, it was deblurred based on strict assumptions, which limited the reconstruction to a small range. We, in contrast, reconstruct the image using a deep neural network. The network is trained only with data simulated using the optical model of our system, i.e., it does not require real-world data that is difficult to obtain. Our trained model generalizes and performs successfully on a variety of moving live objects (e.g., coral polyps, insects) in their natural environment, i.e., it is not specific to a single organism it is trained on. Using the optical model of the system enables recovery of a very large DOF. Our method demonstrates the power of computational photography in expanding the imaging capabilities of lower-cost optics and enables microscopic imaging in dynamic environments, which can reveal important information that is lost when the organisms are captured and examined in the lab. Source Code The code is available on GitHub: https://github.com/VISEAON-Lab/microscopy_deep_dof. Data Sets The file DataSet.zip contains all the datasets needed to train the network (313 ground truth images and 56 kernels). It contains two folders: Kernels and SharpImages. The folder Kernels currently contains the folder ZEMAX_V2_2048, which we used during our experiments. We use this structure to enable running the code using other kernels. The folder ZEMAX_V2_2048 contains one folder named Stacks. The folder Stacks contains two folders: matNorm, which contains the kernels used to blur the images in the simulation, and the folder imgNorm, which contains the same kernels in PNG format. The folder SharpImages contains the 313 sharp images used to generate the simulation, including some that were chosen from the Middlebury dataset [1]. The file RealWorldScenes.zip includes real-world scenes presented in the paper. The first folder contains 25 scenes acquired in the lab environment, and the second folder contains 14 scenes acquired underwater). Each of these folders is divided into the one-shots images (the input to the network) and the restored images that the network outputs. [1] D. Scharstein and C. Pal. Learning conditional random fields for stereo.In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007), Minneapolis, MN, June 2007.
BibTeX
@inproceedings{fischer2022deep,
author = {Judith Fischer and Aviad Avni and Naama Pearl and Amit Peleg and Tali Treibitz},
title = {{Deep Depth-of-Field for Microscopy On-the-Go}},
booktitle = {Zenodo},
year = {2022},
doi = {10.5281/zenodo.6822198},
}Loading related content...