Wide-field, high-resolution reconstruction in computational multi-aperture miniscope using a Fourier neural network.

Qianwan Yang, Ruipeng Guo, Guorong Hu, Yujia Xue, Yunzhe Li, Lei TianOptica 2025 · OpenReview

TL;DR

This paper introduces a computational microscopy system that achieves wide-field, high-resolution imaging with a compact, low-cost miniscope. They developed a novel Fourier neural network, SV-FourierNet, capable of reconstructing images from the miniscope's multiplexed measurements, overcoming limitations of traditional microscopy.

What Problem Does This Paper Address?

Traditional microscopy faces a fundamental trade-off: high resolution typically comes at the cost of a narrow field of view (FOV), and vice-versa. While array microscopes can expand the FOV, they often suffer from scalability issues and complexity. Compact, lensless microscopy methods can reduce contrast and signal-to-noise ratio. This paper tackles these challenges by developing a computational multi-aperture miniscope that uses a microlens array (MLA) to capture wide-field images with high resolution, while also addressing the complex aberrations and view multiplexing inherent in such a system.

Methods

The core innovation is the SV-FourierNet, a multi-channel Fourier neural network designed for image reconstruction. Unlike traditional convolutional neural networks (CNNs) that operate in the spatial domain and can struggle with non-local, spatially-varying aberrations, SV-FourierNet leverages the global processing capabilities of the Fourier transform. This allows it to:

  • Process full-FOV measurements: It can handle large images (12 million pixels) directly, avoiding the need for patch-based processing that can lead to uneven resolution.
  • Learn global receptive fields: By operating in the Fourier domain, it can efficiently capture spatially-varying aberrations across the entire image.
  • Perform blind deconvolution: It learns to invert the imaging process without requiring pre-defined point spread functions (PSFs).
  • Utilize a two-stage approach:
    1. Fourier blind deconvolution module: This module uses learnable Fourier filters to perform initial reconstructions, effectively addressing view multiplexing and aberrations.
  • Enhancement module: A modified Residual Channel Attention Network (RCAN) refines these initial reconstructions to produce the final high-quality image.

The authors also developed a physics-based simulator to generate training data, crucial for training the network without extensive experimental data collection. They also introduced a method to visualize the network's "effective receptive field" (ERF) to understand its learned mapping function and its correlation with physical PSFs.

Datasets

  • Training Data: The network was trained entirely on a physics-based simulator. This simulator generated synthetic data from:
  • Biological samples sourced from public repositories (e.g., 3D whole mouse brain vasculature and neuron datasets).
  • 2D cell images of varying types and morphologies.
  • Synthetically generated fluorescent particles with random sizes, brightness, and locations.
  • Approximately 5000 training images were used.
  • Evaluation Data: The system was evaluated using:
  • Fluorescence resolution targets.
  • Freely moving C. elegans colonies.
  • Weakly scattering mouse brain sections.
  • Fluorescent beads.

Key Insights and Ideas

  • Fourier Domain Learning for Microscopy: Operating in the Fourier domain offers significant advantages for handling complex, spatially-varying aberrations in microscopy, leading to more efficient and robust reconstruction compared to spatial domain learning.
  • ERF as a Tool for Network Interpretation: Visualizing the effective receptive field (ERF) of the neural network provides a direct link between the network's learned function and the physical imaging model, demystifying the "black box" nature of deep learning.
  • Negative ERF for Aberration Suppression: The negative components of the ERF play a crucial role in suppressing aberrations and demultiplexing overlapping signals, contributing to improved image quality and resolution.
  • Physics-Informed Simulation is Key: A well-calibrated, physics-based simulator can generate sufficient high-quality training data, eliminating the need for laborious experimental data collection and enabling robust generalization.

Key Results

  • Uniform Resolution: Achieved a consistent resolution of 7.8 µm across a 6.5 mm FOV.
  • Extended Depth-of-Field (EDOF): Maintained consistent resolution over a 100 µm depth range.
  • Speed: Demonstrated a significant speedup, achieving a 20x speedup compared to CM²Net and a 2x speedup over MultiWienerNet, with inference times as low as 0.06 seconds.
  • Image Quality: Outperformed existing state-of-the-art methods (CM²Net, MultiWienerNet, and LSV ADMM) in terms of PSNR and SSIM, particularly at the FOV edges where other methods degraded.
  • Generalization: The SV-FourierNet, trained solely on simulated data, demonstrated robust generalization to diverse experimental samples, including resolution targets, fluorescent beads, live C. elegans, and brain tissue sections.

Limitations and Future Work

  • Computational Complexity: While SV-FourierNet significantly reduces computational burden compared to spatial domain learning, increasing the number of channels (filters) for even higher resolution can increase computational cost and memory usage.
  • Future Work: The authors suggest extending this principle to 3D imaging applications, potentially removing field stops in Fourier lightfield microscopes to increase FOV without sacrificing resolution. They also anticipate that extending SV-FourierNet to 3D imaging could substantially improve imaging volume with uniform 3D resolution. Further exploration into the sparse nature of system PSFs could lead to further model size and computational load reduction, enabling adaptation to more challenging applications like volumetric shift-variant deconvolution.

Summary updated

BibTeX
@inproceedings{yang2025wide,
  author = {Qianwan Yang and Ruipeng Guo and Guorong Hu and Yujia Xue and Yunzhe Li and Lei Tian},
  title = {{Wide-field, high-resolution reconstruction in computational multi-aperture miniscope using a Fourier neural network.}},
  booktitle = {Optica},
  year = {2025},
  url = {https://openreview.net/forum?id=6LyFPbuVWT},
}
Figure