Abstract
Recently, there has been rapid expansion in the field of micro-connectomics, which targets the three-dimensional (3D) reconstruction of neuronal networks from stacks of two-dimensional (2D) electron microscopy (EM) images. The spatial scale of the 3D reconstruction increases rapidly owing to deep convolutional neural networks (CNNs) that enable automated image segmentation. Several research teams have developed their own software pipelines for CNN-based segmentation. However, the complexity of such pipelines makes their use difficult even for computer experts and impossible for non-experts. In this study, we developed a new software program, called UNI-EM, for 2D and 3D CNN-based segmentation. UNI-EM is a software collection for CNN-based EM image segmentation, including ground truth generation, training, inference, postprocessing, proofreading, and visualization. UNI-EM incorporates a set of 2D CNNs, i.e., U-Net, ResNet, HighwayNet, and DenseNet. We further wrapped flood-filling networks (FFNs) as a representative 3D CNN-based neuron segmentation algorithm. The 2D- and 3D-CNNs are known to demonstrate state-of-the-art level segmentation performance. We then provided two example workflows: mitochondria segmentation using a 2D CNN and neuron segmentation using FFNs. By following these example workflows, users can benefit from CNN-based segmentation without possessing knowledge of Python programming or CNN frameworks.
Introduction: The Technical Barrier in Micro-Connectomics
Mapping the complex wiring of the brain—a field known as micro-connectomics—requires reconstructing 3D neuronal networks from massive stacks of 2D electron microscopy (EM) images. Because these datasets are far too large for manual tracing, researchers rely on deep Convolutional Neural Networks (CNNs) to automate the segmentation of neurons and organelles.
However, a significant "technical wall" exists. Most state-of-the-art CNN tools are fragmented; they require proficiency in Python, knowledge of deep learning frameworks like TensorFlow, and the ability to convert data between obscure file formats. Furthermore, the "pipeline" is often disjointed, requiring different software for painting ground truth, running inference, and proofreading the results. UNI-EM aims to bridge this gap by providing a unified, GUI-driven environment that handles the entire workflow, democratizing access to these powerful tools for biological researchers without programming expertise.
Methods: The UNI-EM Environment
The authors developed UNI-EM, a heterogeneous software environment that combines desktop-based computation with web-based interaction. The architecture is designed to handle the heavy computational load of training while maintaining a responsive user interface for annotation.
A Unified Pipeline
UNI-EM integrates five core stages into a single interface:
- Ground Truth Generation: Utilizing an extended version of "Dojo," a web-based tool supporting multi-scale painting and collaborative proofreading.
- Training: Incorporating a library of 2D CNN architectures (U-Net, ResNet, HighwayNet, and DenseNet) and the 3D Flood-Filling Network (FFN).
- Inference: Running the trained models on large EM volumes.
- Post-processing: Applying filters and connectivity rules to refine segmentation.
- Proofreading and Visualization: Using a new surface mesh-based 3D Annotator.
To ensure accessibility, UNI-EM is provided as a standalone application for Windows and Linux, avoiding the "dependency hell" often associated with Python-based deep learning environments.
Results and Validation
The study validated UNI-EM using the SNEMI3D dataset (mouse somatosensory cortex) and the FIB-25 dataset (fruit fly brain).
Performance and Efficiency
Test users with no prior programming experience achieved impressive results:
- Mitochondria Segmentation (2D): Using a 16-layer ResNet, users achieved a RAND score of 0.85 with only 20 minutes of manual ground truth painting and 10 minutes of training on a consumer-grade GPU (GTX 1070).
- Neuron Segmentation (3D): Using integrated FFNs on the SNEMI3D dataset, a test user achieved a RAND score of 0.84. While specialized labs have achieved higher scores through massive post-processing, UNI-EM provides a high-quality "baseline" accessible to any standard laboratory.
Discussion and Future Work
While UNI-EM significantly lowers the barrier to entry, some challenges remain. Connecting 2D slices into 3D objects currently relies on basic labeling; future versions aim to integrate more sophisticated multicut algorithms. Additionally, while 2D CNNs are highly efficient, 3D Flood-Filling Networks still represent a significant investment, requiring roughly two weeks of computation and substantial 3D ground truth data.
By standardizing data flow using common PNG/TIFF stacks and providing a unified GUI, UNI-EM transforms a complex, expert-only pipeline into a manageable tool for the broader neuroscience community.
Summary updated