Abstract
Image stacks provide invaluable 3D information in various biological and pathological imaging applications. Fourier ptychographic microscopy (FPM) enables reconstructing high-resolution, wide field-of-view image stacks without z-stack scanning, thus significantly accelerating image acquisition. However, existing FPM methods take tens of minutes to reconstruct and gigabytes of memory to store a high-resolution volumetric scene, impeding fast gigapixel-scale remote digital pathology. While deep learning approaches have been explored to address this challenge, existing methods poorly generalize to novel datasets and can produce unreliable hallucinations. This work presents FPM-INR, a compact and efficient framework that integrates physics-based optical models with implicit neural representations (INR) to represent and reconstruct FPM image stacks. FPM-INR is agnostic to system design or sample types and does not require external training data. In our demonstrated experiments, FPM-INR substantially outperforms traditional FPM algorithms with up to a 25-fold increase in speed and an 80-fold reduction in memory usage for continuous image stack representations.
Introduction
In digital pathology, clinicians often require 3D image stacks ($z$-stacks) to examine thick biological samples and complex cellular structures. Fourier Ptychographic Microscopy (FPM) has emerged as a powerful computational imaging technique capable of reconstructing these stacks from a series of low-resolution images captured under varying illumination angles. By synthesizing a high-resolution complex light field from multiple low-resolution measurements, FPM circumvents the physical limitations of standard objectives.
Despite its potential, FPM faces a significant "computational bottleneck." Traditional iterative algorithms reconstruct each slice of a 3D volume independently, a process that is computationally intensive—often taking tens of minutes—and results in massive data storage requirements, reaching gigabytes per sample. While deep learning has been proposed to mitigate these issues, standard "black-box" models often struggle with generalization across different microscope configurations or tissue types and are prone to generating unreliable artifacts or "hallucinations."
Methodology: Implicit Neural Representations for Microscopy
FPM-INR addresses these challenges by shifting the reconstruction paradigm from discrete voxel storage to learning a continuous function. The framework integrates several key innovations:
- Implicit Neural Representations (INR): Instead of a 3D grid of voxels, FPM-INR employs a compact Multi-Layer Perceptron (MLP) to represent the sample's light field. Given a coordinate $(x, y, z)$, the network outputs the predicted amplitude and phase at that specific point.
- Physics-Informed Self-Supervision: Unlike traditional deep learning, FPM-INR does not require a pre-trained database. It utilizes the known physics of light propagation (the FPM forward model) as a self-supervisory signal. The system optimizes the neural network by comparing the microscope's actual raw measurements with estimated measurements generated by passing the network's output through the physical model.
- Low-Rank Feature Space: To optimize memory efficiency, the authors leverage the inherent anisotropy of microscopy data, where lateral resolution ($x, y$) is typically much higher than axial resolution ($z$). They represent the volume using a 2D feature plane and a 1D feature vector, a tensor decomposition that captures structural details with significantly fewer parameters.
- Strategic Optimization: The training process alternates between uniform and random $z$-slice selection, preventing the model from converging to local minima and accelerating the reconstruction of a continuous volume.
Experimental Results and Performance
The framework was validated on challenging biological targets, including human blood smears and thyroid gland lesions. In the blood smear experiments, a slide was tilted at a 4-degree angle to create a complex 3D scene. FPM-INR demonstrated a massive reduction in data volume, compressing the representation from 2576 MB to just 32 MB—an 80-fold reduction. Furthermore, the reconstruction process was up to 25 times faster than traditional iterative algorithms on identical hardware.
Qualitatively, FPM-INR produced higher-fidelity images with fewer artifacts, particularly at large defocus distances where traditional FPM struggles. In thyroid cytology specimens, which were approximately 30 $\mu$m thick, FPM-INR successfully resolved dense cell aggregations that were otherwise blurred or distorted in traditional reconstructions.
Discussion and Conclusion
FPM-INR represents a significant advancement in volumetric microscopy, offering a system-agnostic solution that combines the reliability of physics-based models with the efficiency of neural representations. By learning a continuous function, it enables smooth, arbitrary-depth visualization without the constraints of discrete slicing. While current inference speeds are high, future advancements in GPU hardware and specialized encoding techniques, such as hash-encoding, are expected to further enhance the real-time capabilities of this framework for gigapixel-scale digital pathology.
Summary updated