Advanced Deep Networks for 3d Mitochondria Instance Segmentation

Mingxing Li, Chang Chen, Xiaoyu Liu, Wei Huang, Yueyi Zhang, Zhiwei Xiong

TL;DR

Researchers from the University of Science and Technology of China developed two specialized deep residual networks, Res-UNet-R and Res-UNet-H, to tackle the complex task of segmenting individual mitochondria in 3D electron microscopy (EM) volumes. By introducing a novel anisotropic convolution block and a dual-path decoder for noisy human samples, their approach secured 1st place in the ISBI 2021 MitoEM Challenge.

What Problem Does This Paper Address?

Mitochondria are the "powerhouses" of the cell, and mapping their structure in 3D is vital for biological research. However, electron microscopy (EM) produces massive datasets (often in the Terabyte range) where mitochondria appear as elongated, distorted, and densely packed structures.

Manual segmentation is impossible at this scale. While deep learning has improved automation, two major hurdles remain:

  1. Anisotropy: EM data often has different resolutions across axes (e.g., high resolution within a slice, lower resolution between slices), which confuses standard 3D kernels.
  2. Noise and Complexity: Human tissue samples are significantly noisier and more complex than animal models (like rats), making it difficult for a single model to perform well across different species.

Methods

The authors propose a bottom-up segmentation framework centered on two new architectures:

  • Res-UNet-R (for Rat samples): A residual U-Net that predicts a semantic mask and instance boundaries simultaneously using a single decoder path.
  • Res-UNet-H (for Human samples): To handle the higher noise levels in human data, this version uses dual decoder paths—one dedicated to the semantic mask and another to the instance boundary.
  • Anisotropic Convolution Block (ACB): Instead of standard 3D convolutions, the ACB uses a $1 \times 3 \times 3$ layer followed by cascaded $3 \times 3 \times 3$ layers. This specifically targets the non-uniform resolution of the MitoEM dataset.
  • Multi-scale Training (MT): The models are trained in two stages, starting with smaller input sizes ($32 \times 256 \times 256$) and fine-tuning with larger ones ($36 \times 320 \times 320$) to increase the receptive field and prevent overfitting.
  • Denoising Pre-processing: For the noisy human samples, the team utilized an interpolation network that uses adjacent frames to "restore" noisy regions before they ever reach the segmentation model.

Datasets

The primary benchmark is the MitoEM dataset, which contains two $30 \mu m^3$ EM image volumes: one from rat tissue (MitoEM-R) and one from human tissue (MitoEM-H). They also validated their approach on the Lucchi dataset, a standard benchmark for mitochondrial semantic segmentation.

Key Insights and Ideas

  • Species-Specific Architectures: The most striking insight is that "one size does not fit all." The authors found that the increased noise in human samples required a structural change (the dual-path decoder) rather than just more training data.
  • Boundary-Aware Seeds: Instead of just predicting "is this a mitochondrion?", the model predicts boundaries. By subtracting the predicted boundary from the semantic mask, they create "seeds" that make it much easier to separate two mitochondria that are touching.
  • The Power of Denoising: Simply adding a denoising step as pre-processing significantly boosted performance on the human test set, highlighting that even the best architectures struggle when the input signal-to-noise ratio is low.

Key Results

The proposed method achieved state-of-the-art results across the board:

  • ISBI 2021 Challenge: Ranked 1st place overall.
  • MitoEM Validation: Achieved an AP-75 score of 0.917 on Rat samples and 0.828 on Human samples, significantly outperforming previous leaders like Li et al. (0.870 and 0.787 respectively).
  • Lucchi Dataset: Achieved a Jaccard index of 0.895 and a Dice score of 0.945, proving the model's versatility for both instance and semantic segmentation tasks.

Limitations and Future Work

While the dual-decoder approach for human samples was successful, the authors noted that human samples remain significantly more challenging than rat samples. The current denoising step relies on an interpolation network that requires adjacent frames; future work might explore more integrated, end-to-end denoising within the segmentation network itself. Additionally, while the ACB handles anisotropy well, the specific kernel sizes are currently hand-tuned for the MitoEM resolution and might need adjustment for different EM hardware setups.

Summary updated

Abstract

Mitochondria instance segmentation from electron microscopy (EM) images has seen notable progress since the introduction of deep learning methods. In this paper, we propose two advanced deep networks, named Res-UNet-R and Res-UNet-H, for 3D mitochondria instance segmentation from Rat and Human samples. Specifically, we design a simple yet effective anisotropic convolution block and deploy a multi-scale training strategy, which together boost the segmentation performance. Moreover, we enhance the generalizability of the trained models on the test set by adding a denoising operation as pre-processing. In the Large-scale 3D Mitochondria Instance Segmentation Challenge at ISBI 2021, our method ranks the 1st place. Code is available at https://github.com/Limingxing00/MitoEM2021-Challenge.
BibTeX
@inproceedings{li2022advanced,
  author = {Mingxing Li and Chang Chen and Xiaoyu Liu and Wei Huang and Yueyi Zhang and Zhiwei Xiong},
  title = {{Advanced Deep Networks for 3d Mitochondria Instance Segmentation}},
  booktitle = {ISBI},
  year = {2022},
  url = {https://openreview.net/forum?id=OAlUZS_JKAZR},
}

Loading related content...

Figure