Neural Network Segmentation of Cell Ultrastructure Using Incomplete Annotation

John Paul Francis, Hongzhi Wang, Kate L. White, Tanveer Syeda-Mahmood, Raymond C. StevensarXiv 2020 · DOI · arXiv

Abstract

The Pancreatic beta cell is an important target in diabetes research. For scalable modeling of beta cell ultrastructure, we investigate automatic segmentation of whole cell imaging data acquired through soft X-ray tomography. During the course of the study, both complete and partial ultrastructure annotations were produced manually for different subsets of the data. To more effectively use existing annotations, we propose a method that enables the application of partially labeled data for full label segmentation. For experimental validation, we apply our method to train a convolutional neural network with a set of 12 fully annotated data and 12 partially annotated data and show promising improvement over standard training that uses fully annotated data alone.

Introduction

In structural biology, the principle that "structure determines function" is paramount. Understanding diseases like diabetes requires high-resolution 3D maps of pancreatic beta cells, often generated using soft X-ray tomography. However, the manual identification of cellular components—such as the nucleus, mitochondria, and insulin vesicles—presents a significant bottleneck. Expert segmentation of a single cell can take up to eight hours, leading to inconsistent datasets where early data may only feature coarse labels (e.g., nucleus and membrane) while later data includes fine-grained details. Standard machine learning approaches typically discard incomplete data or default to the "lowest common denominator" of labels. This research addresses how to utilize all available data, regardless of label completeness.

Methods

The proposed training scheme leverages both fully and partially labeled data simultaneously through three key components:

  1. Label Mapping ($T^P$): A mapping function translates fine-grained labels into coarse labels. In the partially labeled set, mitochondria and insulin vesicles are "absorbed" into the membrane category. The function formally maps: mitochondria + insulin vesicle + membrane (fine) = membrane (coarse).
  2. Modified Cross-Entropy Loss: During training, fully labeled images use standard loss. For partially labeled images, a modified loss function sums the model's probabilities for all sub-categories belonging to a specific coarse label, preventing the model from being penalized for predicting a fine-grained structure that is unlabeled in the coarse set.
  3. Architecture: A 2D U-Net was employed. Despite the 3D nature of the data, a 2D approach was chosen to bypass memory constraints of 3D convolutions, maintaining the high resolution necessary for detecting small insulin vesicles.

Datasets

The study utilized 3D grayscale tomograms (~25nm³/voxel resolution) from three cell lines:

  • INS-1E (Full Annotation): 27 tomograms with membrane, nucleus, mitochondria, and insulin vesicles manually identified.
  • 1.1B4 & HEK (Partial Annotation): 12 tomograms where only the membrane and nucleus were labeled, with mitochondria and vesicles semantically merged into the membrane label.

Results and Discussion

The "Mixed Labels" approach significantly outperformed the baseline model trained only on fully annotated data. Quantitatively, the average Dice Similarity Coefficient (DSC) improved from 0.639 to 0.706, representing a 10.5% relative increase.

Table 1: Quantitative comparison of DSC scores between Baseline and Mixed Labels models across different cell structures.

Notably, mitochondria segmentation saw the largest improvement at 23.1%. This suggests that adding data without explicit mitochondria labels helped the model learn the general cellular context and boundaries, improving its ability to localize specific organelles. Qualitatively, the mixed-label model demonstrated greater robustness to varying backgrounds and produced cleaner internal structures compared to the baseline.

Figure 1: Qualitative comparison of segmentation results across three sample images. The 'ours' column (Mixed Labels) shows closer alignment with manual ground truth than the baseline.

Conclusion

This method demonstrates that older or less detailed annotations are valuable assets rather than legacy noise. By accounting for different "semantic depths" in labels, researchers can scale whole-cell modeling more effectively. Future work will explore 3D CNN architectures and scaling the unified model to incorporate a wider variety of datasets with diverse labeling schemes.

Summary updated

Abstract

The Pancreatic beta cell is an important target in diabetes research. For scalable modeling of beta cell ultastructure, we investigate automatic segmentation of whole cell imaging data acquired through soft X-ray tomography. During the course of the study, both complete and partial ultrastrucutre annotations were produced manually for different subsets of the data. To more effectively use existing annotations, we propose a method that enables the application of partially labeled data for full label segmentation. For experimental validation, we apply our method to train a convolutional neural network with a set of 12 fully annotated data and 12 partially annotated data and show promising improvement over standard training that uses fully annotated data alone.
BibTeX
@misc{francis2020neural,
  author = {John Paul Francis and Hongzhi Wang and Kate L. White and Tanveer Syeda-Mahmood and Raymond C. Stevens},
  title = {{Neural Network Segmentation of Cell Ultrastructure Using Incomplete Annotation}},
  year = {2020},
  eprint = {2004.09673},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.09673},
}
Figure