Mimicry embedding for advanced neural network training of 3D biomedical micrographs

Artur Yakimovich, Moona Huttunen, Jerzy Samolej, Barbara Clough, Nagisa Yoshida, Serge Mostowy, Eva‐Maria Frickel, Jason MercerbioRxiv 2019 · DOI

Abstract: The use of deep neural networks (DNNs) for analysis of complex biomedical images shows great promise but is hampered by a lack of large verified datasets for rapid network evolution. Here we present a novel “mimicry embedding” strategy for rapid application of neural network architecture-based analysis of biomedical imaging datasets. Embedding of a novel biological dataset, such that it mimics a verified dataset, enables efficient deep learning and seamless architecture switching. We apply this strategy across various microbiological phenotypes; from super-resolved viruses to in vivo parasitic infections. We demonstrate that mimicry embedding enables efficient and accurate analysis of three-dimensional microscopy datasets. The results suggest that transfer learning from pre-trained network data may be a powerful general strategy for analysis of heterogeneous biomedical imaging datasets.

Introduction

Deep learning has revolutionized image analysis, but it faces a significant "data hunger" problem in biology. While computer vision benefits from massive, verified datasets like ImageNet or MNIST, biological data is often small, heterogeneous, and difficult to verify. Standard Convolutional Neural Networks (CNNs) also struggle with spatial relationships due to "pooling" layers that discard precise positional information. Capsule Networks (CapsNet) address this but typically require vast amounts of training data.

Methods: ZedMate and Mimicry Embedding

The researchers developed a workflow centered on two main components:

  1. ZedMate: An ImageJ-Fiji plugin that detects and links individual particles (e.g., viruses or parasites) across 3D Z-stacks, converting fluorescent signals into grayscale intensity matrices.
  2. Mimicry Embedding: This core innovation reshapes 3D intensity profiles into 28x28 pixel grayscale images using linear interpolation and padding. These images "mimic" the MNIST handwritten digit dataset format.
ZedMate particle detection and Z-Intensities Matrix for mimicry embedding.

By mapping biological data into the MNIST feature space, the team leveraged transfer learning. They pre-trained a CapsNet on a binary version of MNIST and then transferred those weights to biological classification tasks, bypassing the need for massive manual annotation.

Datasets

The approach was validated across several datasets:

  • MNIST: 70,000 handwritten digits for initial weight training.
  • Vaccinia Virus (VACV): 3D images used to distinguish "cell-free" from "cell-associated" virions.
  • Toxoplasma gondii (Tg): 3D images used to classify "viable" vs. "unviable" parasites using ubiquitin as a weak label.
  • In vivo Zebrafish: 3D stereomicroscopy tracking Tg-EGFP infection progression in living larvae.

Key Insights

  • The "Mimicry" Shortcut: Mapping biological data to existing feature spaces like MNIST avoids the bottleneck of manual annotation.
  • CapsNet for Biology: Capsule Networks are ideal for microscopy as they preserve the relative orientation and spatial arrangement of features.
  • Weak Labeling: Using specific markers (like ubiquitin) as "weak labels" allows for automated training data annotation, which can then be removed during inference.

Key Results

The mimicry-embedded CapsNet achieved 96.5% accuracy in separating cell-free from cell-associated Vaccinia viruses, significantly outperforming manual binning. It also achieved 82% accuracy in differentiating intra- from extracellular virions without specific antibody labeling.

Visualization of automated particle detection and binning results.

When faced with more complex Toxoplasma phenotypes, the researchers seamlessly switched to a DropConnect architecture using the same mimicry embedding, demonstrating the strategy's flexibility. In vivo, the system successfully tracked parasite clearance in zebrafish, matching known biological outcomes with high accuracy.

Limitations and Future Work

A >10% drop in accuracy was observed when applying the simplest mimicry strategy to highly complex Toxoplasma phenotypes, suggesting that more complex "landmark" biological datasets might be needed for subtler features. Future work will explore expanding this "common denominator" approach to other 3D modalities like cryo-electron tomography or lattice light-sheet microscopy.

Summary updated

Abstract

The use of deep neural networks (DNNs) for analysis of complex biomedical images shows great promise but is hampered by a lack of large verified datasets for rapid network evolution. Here we present a novel “mimicry embedding” strategy for rapid application of neural network architecture-based analysis of biomedical imaging datasets. Embedding of a novel biological dataset, such that it mimics a verified dataset, enables efficient deep learning and seamless architecture switching. We apply this strategy across various microbiological phenotypes; from super-resolved viruses to in vivo parasitic infections. We demonstrate that mimicry embedding enables efficient and accurate analysis of three-dimensional microscopy datasets. The results suggest that transfer learning from pre-trained network data may be a powerful general strategy for analysis of heterogeneous biomedical imaging datasets.
BibTeX
@misc{yakimovich2019mimicry,
  author = {Artur Yakimovich and Moona Huttunen and Jerzy Samolej and Barbara Clough and Nagisa Yoshida and Serge Mostowy and Eva‐Maria Frickel and Jason Mercer},
  title = {{Mimicry embedding for advanced neural network training of 3D biomedical micrographs}},
  year = {2019},
  doi = {10.1101/820076},
  howpublished = {bioRxiv},
}
Figure