Abstract: The use of deep neural networks (DNNs) for analysis of complex biomedical images shows great promise but is hampered by a lack of large verified datasets for rapid network evolution. Here we present a novel “mimicry embedding” strategy for rapid application of neural network architecture-based analysis of biomedical imaging datasets. Embedding of a novel biological dataset, such that it mimics a verified dataset, enables efficient deep learning and seamless architecture switching. We apply this strategy across various microbiological phenotypes; from super-resolved viruses to in vivo parasitic infections. We demonstrate that mimicry embedding enables efficient and accurate analysis of three-dimensional microscopy datasets. The results suggest that transfer learning from pre-trained network data may be a powerful general strategy for analysis of heterogeneous biomedical imaging datasets.
Introduction
Deep learning has revolutionized image analysis, but it faces a significant "data hunger" problem in biology. While computer vision benefits from massive, verified datasets like ImageNet or MNIST, biological data is often small, heterogeneous, and difficult to verify. Standard Convolutional Neural Networks (CNNs) also struggle with spatial relationships due to "pooling" layers that discard precise positional information. Capsule Networks (CapsNet) address this but typically require vast amounts of training data.
Methods: ZedMate and Mimicry Embedding
The researchers developed a workflow centered on two main components:
- ZedMate: An ImageJ-Fiji plugin that detects and links individual particles (e.g., viruses or parasites) across 3D Z-stacks, converting fluorescent signals into grayscale intensity matrices.
- Mimicry Embedding: This core innovation reshapes 3D intensity profiles into 28x28 pixel grayscale images using linear interpolation and padding. These images "mimic" the MNIST handwritten digit dataset format.
By mapping biological data into the MNIST feature space, the team leveraged transfer learning. They pre-trained a CapsNet on a binary version of MNIST and then transferred those weights to biological classification tasks, bypassing the need for massive manual annotation.
Datasets
The approach was validated across several datasets:
- MNIST: 70,000 handwritten digits for initial weight training.
- Vaccinia Virus (VACV): 3D images used to distinguish "cell-free" from "cell-associated" virions.
- Toxoplasma gondii (Tg): 3D images used to classify "viable" vs. "unviable" parasites using ubiquitin as a weak label.
- In vivo Zebrafish: 3D stereomicroscopy tracking Tg-EGFP infection progression in living larvae.
Key Insights
- The "Mimicry" Shortcut: Mapping biological data to existing feature spaces like MNIST avoids the bottleneck of manual annotation.
- CapsNet for Biology: Capsule Networks are ideal for microscopy as they preserve the relative orientation and spatial arrangement of features.
- Weak Labeling: Using specific markers (like ubiquitin) as "weak labels" allows for automated training data annotation, which can then be removed during inference.
Key Results
The mimicry-embedded CapsNet achieved 96.5% accuracy in separating cell-free from cell-associated Vaccinia viruses, significantly outperforming manual binning. It also achieved 82% accuracy in differentiating intra- from extracellular virions without specific antibody labeling.
When faced with more complex Toxoplasma phenotypes, the researchers seamlessly switched to a DropConnect architecture using the same mimicry embedding, demonstrating the strategy's flexibility. In vivo, the system successfully tracked parasite clearance in zebrafish, matching known biological outcomes with high accuracy.
Limitations and Future Work
A >10% drop in accuracy was observed when applying the simplest mimicry strategy to highly complex Toxoplasma phenotypes, suggesting that more complex "landmark" biological datasets might be needed for subtler features. Future work will explore expanding this "common denominator" approach to other 3D modalities like cryo-electron tomography or lattice light-sheet microscopy.
Summary updated