TL;DR
This paper introduces 2PiLnet, a deep learning framework that uses light-field microscopy (LFM) to achieve high-speed, volumetric calcium imaging in scattering brain tissue. By combining LFM's scanless nature with a physics-informed neural network, 2PiLnet reconstructs neural activity with two-photon-like contrast and reduced scattering artifacts, enabling real-time analysis for complex neuroscience experiments.
What Problem Does This Paper Address?
Understanding neural circuit function requires observing the activity of many neurons simultaneously over time. Traditional methods like two-photon (2P) microscopy offer high-quality imaging but are slow due to scanning. Light-field microscopy (LFM) offers a solution by capturing entire volumes in a single snapshot, dramatically increasing speed and throughput. However, LFM suffers from two major drawbacks: its high computational cost for volume reconstruction and its susceptibility to light scattering in biological tissues, which degrades image quality and signal-to-noise ratio (SNR). These limitations have hindered LFM's widespread adoption for real-time, adaptive experiments that demand fast, reliable neural activity data from scattering environments like the mammalian brain.
Methods
The core innovation is the 2PiLnet deep learning framework. It's a physics-based neural network designed to reconstruct high-quality volumetric calcium signals from one-photon LFM data, even in the presence of scattering.
Here's a breakdown of the approach:
- Dual-Mode Microscopy: The system combines a standard two-photon (2P) laser scanning microscope with a light-field microscope (LFM). This allows for acquiring both high-contrast 2P data (for training) and fast, scanless LFM data (for actual experiments).
- 2PiLnet Architecture: This physics-informed neural network is trained using a combination of 2P volumes and one-photon light fields. It integrates the LFM wave-optics forward model into its training process, ensuring more reliable volume estimation.
- Training Strategy: 2PiLnet undergoes a two-phase training:
- Supervised Training: Uses paired 2P z-stacks and corresponding LFM z-stacks of structural markers (tdTomato) to learn the relationship between scattered light-field data and high-quality 2P volumes.
- Self-Supervised Training: Leverages LF videos of calcium indicators (jGCaMP8f) and the LFM forward model to refine volume estimation without requiring additional 2P data for every new field of view.
- Calcium Signal Extraction:
- Preprocessing: Raw LF videos are processed to create an "LF activity map" that highlights active neurons with high contrast.
- Segmentation: The 2PiLnet-estimated volume is segmented into regions of interest (ROIs) using Voronoi-Otsu labeling.
- Time Series Calculation: Two methods are used:
- 2PiLnet ROI: Calculates the mean intensity within each ROI over time.
- 2PiLnet Matrix: Uses a CNN to estimate each neuron's "footprint" in the light field, forming a matrix that's used with the LF video to compute time series via matrix factorization.
Datasets
- Training Data:
- Paired 2P z-stacks and one-photon LFM z-stacks of tdTomato-expressing neurons in mouse neocortical brain slices.
- One-photon LFM videos of jGCaMP8f-expressing neurons in mouse neocortical brain slices.
- Evaluation Data:
- Light-field videos of jGCaMP8f-expressing neurons in neocortical brain slices, acquired at 100 Hz.
- Simulated LF videos of neuronal calcium activity corrupted by scattering and noise (for assessing precision and recall).
The paper does not introduce a new benchmark dataset but rather uses existing experimental setups and data types for training and evaluation.
Key Insights and Ideas
- Bridging the Gap: 2PiLnet successfully bridges the gap between the speed of LFM and the quality of 2P imaging, offering a path to high-throughput, scattering-robust neural circuit analysis.
- Physics-Informed AI: Integrating the physics of LFM (wave-optics forward model) into the deep learning architecture is crucial for reliable volume reconstruction and generalization to unseen data.
- Generalization Power: The trained 2PiLnet model can generalize to new fields of view without requiring new 2P ground truth data, significantly reducing experimental overhead.
- Scattering Mitigation: The network effectively reconstructs volumes with two-photon-like contrast and source confinement, even from scattered, blurry one-photon LFM inputs.
- Real-time Potential: The dramatic reduction in processing time (hours to minutes) brings volumetric LFM analysis within reach of single-workstation, real-time applications.
Key Results
- High-Speed Imaging: Achieved volumetric imaging at 100 volumes per second.
- Scattering Mitigation: Reconstructed volumes with two-photon-like contrast and source confinement from scattered LFM data.
- High SNR: Extracted calcium signals with high signal-to-noise ratios (median SNR ~13 dB for 2PiLnet Matrix and ROI).
- Reduced Crosstalk: Significantly reduced optical crosstalk between adjacent neurons compared to conventional methods like Richardson-Lucy (RL) deconvolution and seeded-iterative demixing (SID). For example, the correlation coefficient between adjacent cells was reduced from 0.60 (RL) to 0.41 (2PiLnet ROI).
- Spike Detection: Enabled the detection of putative spikes fired at rates up to 10 Hz.
- Computational Speedup: Reduced LF video processing time from hours to minutes, an ~18-fold reduction compared to SID for the 2PiLnet Matrix workflow.
- Neuron Ensemble Mapping: Demonstrated the ability to map spatially intermingled neuronal ensembles with correlated activity.
Limitations and Future Work
- Depth Limitations: The study primarily imaged the superficial 100 microns of brain slices. Future work will assess 2PiLnet's performance at greater depths in vivo.
- Motion Artifacts: In vivo imaging will require correction for motion artifacts, which are not present in the in vitro slice preparations.
- Generalizability to Different Optics/Samples: While 2PiLnet generalizes to unseen FOVs, further characterization is needed for its performance with different LFM optics, system-to-system alignment variations, and samples with varying indicator sparsities and depths.
- Hardware Advancements: The camera's maximum frame rate currently limits the volume rate. Faster, more sensitive cameras could enable even higher imaging speeds.
- Real-time Processing: While processing is significantly faster, achieving true real-time signal extraction for adaptive experiments may require further optimization, potentially through GPU or FPGA architectures.
- Training Data Requirements: While 2PiLnet requires only a single 2P volume for initial training, future work could explore training with even less data or adapting to different microscope configurations without 2P data.
Summary updated