Abstract
This work presents a unified self-supervised neural network framework, VUSMamba, which offers a standardized pipeline for constructing and analyzing whole-brain cell-type atlases. By leveraging the efficient Mamba architecture and pre-training on large-scale unlabeled volumetric data, the system enables high-precision cell segmentation in thick (300µm) brain slices while significantly reducing the need for manual annotations and computational overhead.
Introduction
Understanding the 3D spatial distribution of diverse cell types is fundamental to neuroscience. While multiplexed fluorescence in situ hybridization (multi-FISH) allows for the molecular labeling of specific gene expressions, analyzing the resulting volumetric data is challenging. Standard brain slices are typically thin (10µm), losing critical 3D context. Moving to "thick" slices (300µm) preserves this context but generates massive datasets that are difficult to annotate manually and computationally expensive to process using traditional Transformer-based architectures.
Methods and Pipeline
The researchers developed the Voxelwise U-shaped Swin-Mamba (VUSMamba) network to address these bottlenecks. The framework integrates a specialized imaging and computational pipeline:
- Imaging Pipeline: High-speed volumetric microscopy (VISoR) is combined with multi-FISH labeling to capture molecular markers across thick tissue sections.
- Mamba Architecture: Unlike Transformers, which suffer from quadratic complexity, VUSMamba utilizes State Space Models (SSMs). This allows for linear complexity when processing long sequences of 3D image data, making whole-brain analysis memory-efficient.
- Self-Supervised Learning (SSL): To bypass the "annotation bottleneck," the model is pre-trained on 13,736 unlabeled volumes. It learns anatomical structures through three pretext tasks: rotation prediction, image reconstruction, and context-aware recovery. This pre-training allows the model to achieve high performance with only a small set of expert-annotated samples for fine-tuning.
Key Results
The VUSMamba framework was evaluated using markers for all cell nuclei (Hoechst), excitatory glutamatergic neurons (Vglut1), and inhibitory GABAergic neurons (Vgat).
- High Accuracy: The model achieved Dice scores between 0.89 and 0.93 across different markers, outperforming state-of-the-art models like U-Net and SwinUNETR.
- Biological Insights: The system successfully mapped neuronal densities. For instance, it revealed that the CA1 region of the hippocampus maintains a nearly 1:1 ratio of excitatory to inhibitory neurons, while the barrel cortex (SSp-bfd) is dominated by excitatory neurons (~60%).
- Boundary Refinement: The study found that molecularly defined cell-type boundaries often differ from classical anatomical boundaries in the Allen Mouse Brain Atlas, suggesting a need for more nuanced, molecularly-driven brain maps.
Discussion and Future Work
VUSMamba demonstrates that self-supervised learning can effectively unlock the potential of large-scale volumetric brain data. By reducing the reliance on manual labels, this approach can be extended to map rare cell types. Future work will focus on scaling the system to handle dozens of markers simultaneously and addressing the infrastructure challenges posed by petabyte-scale whole-brain datasets.
Summary updated