Learning cellular morphology with neural networks

Philipp J. Schubert, Sven Dorkenwald, Michał Januszewski, Viren Jain, Joergen Kornfeld

TL;DR

This paper introduces Cellular Morphology Neural Networks (CMNs), a novel approach for analyzing the intricate 3D structures of neurons from electron microscopy data. By leveraging multi-view 2D projections and unsupervised "Neuron2vec" embeddings, CMNs can automatically identify cell types, subcellular compartments, and even correct segmentation errors, significantly advancing the automation of neural circuit analysis.

What Problem Does This Paper Address?

Analyzing the complex 3D architecture of neurons from massive electron microscopy datasets is a monumental challenge. While automated neuron reconstruction and synapse detection have seen significant progress, the detailed morphological analysis of these nanometer-resolution reconstructions remains largely manual. This gap hinders our ability to understand neuronal circuits and their functions. The paper aims to bridge this gap by developing automated methods for morphological analysis at the cellular and subcellular levels.

Methods

The core of the paper is the Cellular Morphology Neural Network (CMN). CMNs are designed to analyze multi-channel 2D projections of 3D neuronal reconstructions. Key methodological contributions include:

  • Multi-view Projections: Instead of processing dense 3D volumes, CMNs utilize sampled 2D projections from different viewpoints. This approach balances the need for high-resolution detail with computational feasibility, especially for large and sparse neuronal structures.
  • Unsupervised "Neuron2vec" Embeddings: The authors introduce "Neuron2vec," a method using unsupervised triplet-loss training to learn low-dimensional embeddings of neuronal morphology. This allows for unbiased morphological comparisons and database queries without relying on hand-crafted features.
  • Supervised Classification: CMNs are trained for supervised tasks such as identifying glia cells, classifying neuronal cell types, and segmenting subcellular compartments (axon, dendrite, soma).
  • Top-Down Error Correction: A novel strategy is proposed to resolve false mergers in neuron reconstructions by leveraging glia cell classification.
  • High-Resolution Semantic Segmentation: For fine-grained analysis, CMNs are extended to perform sub-micron resolution segmentation of cellular surfaces, such as identifying dendritic spines.

Datasets

The study utilizes a large electron microscopy dataset from the zebra finch brain (Area X). This dataset consists of:

  • FFN-generated meshes and skeletons: These form the basis for neuron reconstructions.
  • Manually annotated data: Used for training and validating CMNs for various tasks, including glia identification, cell type classification, and subcellular compartment identification. Specific datasets are mentioned for each task, often derived from subsets of the main EM data.
  • A benchmark for glia classification: The paper presents results on a dataset of 118 manually annotated glia SVs.
  • A benchmark for cell type classification: This involves 402 manually traced cells.
  • A benchmark for subcellular compartment identification: This uses 33 neurites with manually annotated compartments.

Key Insights and Ideas

  • Multi-view projections are powerful: The paper demonstrates that 2D projections, when sampled intelligently, can capture sufficient morphological information for complex 3D analysis, outperforming some 3D-native approaches.
  • Unsupervised embeddings unlock new analysis: "Neuron2vec" embeddings provide a generalizable way to represent neuronal morphology, enabling comparisons and queries without prior feature engineering.
  • Glia detection is a key to error correction: Identifying glia cells effectively can be used to resolve false mergers in neuron reconstructions, a common problem in automated pipelines.
  • Cell organelles are crucial for fine-grained classification: Information from cell organelles (like mitochondria and synaptic junctions) significantly improves the accuracy of cell type and compartment identification, especially for reconstructions with limited context.
  • CMNs generalize well: The approach shows promise for various tasks, from broad cell type classification to fine-grained surface segmentation.

Key Results

  • Glia Classification: Achieved an average F1-score of 0.979 for glia cell identification.
  • Neuron Type Classification: CMNs achieved a weighted F1-score of 0.987 for cell type classification using 10 views, outperforming previous methods.
  • Subcellular Compartment Identification: CMNs achieved an F1-score of 0.996 on the validation set for identifying axons, dendrites, and somata.
  • Dendritic Spine Segmentation: Demonstrated excellent performance with an F1-score of 0.978 for identifying dendritic spines.
  • Error Correction: The glia splitting heuristic significantly improved classification performance, with a weighted F1-score of 0.995.

Limitations and Future Work

  • Computational Cost: While more efficient than dense 3D methods, rendering and processing multi-view projections can still be computationally intensive.
  • Data Dependency: The performance of CMNs is dependent on the quality and completeness of the underlying neuron reconstructions.
  • Generalization to Other Datasets: While the approach is presented as generalizable, its performance on different EM datasets with varying staining and reconstruction quality is not fully explored.
  • Future Work: The authors suggest several avenues:
  • Comparing triplet-loss embeddings with other unsupervised methods like autoencoders.
  • Integrating CMN classification results into graph-cut segmentation algorithms.
  • Using CMN outputs to directly evaluate shape plausibility and detect reconstruction errors.
  • Exploring alternative 3D network architectures (e.g., PointNets) for comparison.
  • Investigating the impact of lower-resolution, zoomed-out views.

Summary updated

Abstract

Abstract Reconstruction and annotation of volume electron microscopy data sets of brain tissue is challenging, but can reveal invaluable information about neuronal circuits. Significant progress has recently been made in automated neuron reconstruction, as well as automated detection of synapses. However, methods for automating the morphological analysis of nanometer-resolution reconstructions are less established, despite their diverse application possibilities. Here, we introduce cellular morphology neural networks (CMNs), based on multi-view projections sampled from automatically reconstructed cellular fragments of arbitrary size and shape. Using unsupervised training we inferred morphology embeddings (“Neuron2vec”) of neuron reconstructions and trained CMNs to identify glia cells in a supervised classification paradigm which was then used to resolve neuron reconstruction errors. Finally, we demonstrate that CMNs can be used to identify subcellular compartments and the cell types of neuron reconstructions.
BibTeX
@inproceedings{schubert2019learning,
  author = {Philipp J. Schubert and Sven Dorkenwald and Michał Januszewski and Viren Jain and Joergen Kornfeld},
  title = {{Learning cellular morphology with neural networks}},
  booktitle = {Nature Communications},
  year = {2019},
  url = {https://openreview.net/forum?id=06JwAJXpC1},
}
Figure