Self-Supervised Voxel-Level Representation Rediscovers Subcellular Structures in Volume Electron Microscopy

Hongqing Han, Mariia Dmitrieva, Alexander Sauer, Ka Ho Tam, Jens RittscherWorkshops @ CVPR 2022 · DOI

TL;DR

This paper introduces a novel self-supervised learning approach to automatically segment subcellular structures in large, high-resolution 3D electron microscopy volumes. By learning a "semantic" voxel-level representation that is invariant to transformations, the method effectively distinguishes and separates major cellular components without requiring any manual annotations.

What Problem Does This Paper Address?

Modern biological imaging techniques like FIB-SEM are generating massive, high-resolution 3D datasets of cells. Extracting meaningful biological insights from this deluge of data is challenging, especially when it comes to understanding the intricate organization of subcellular structures. Traditional methods often rely on manual annotation or supervised learning, which are time-consuming and don't scale well with the increasing data volume. While previous work has focused on learning representations for cropped images of single cells or molecules, a robust and generalizable method for learning voxel-level representations of entire cellular volumes for unsupervised segmentation has been lacking. This paper aims to fill that gap by developing a self-supervised approach that can automatically identify and delineate subcellular structures within these complex 3D datasets.

Methods

The core of the proposed method is a Variational Auto-Encoder (VAE) combined with metric learning. The VAE learns to encode small 3D image patches (representing voxels and their surroundings) into a latent space. The key innovation lies in how this latent space is structured:

  • Disentangled Latent Space: The authors propose separating the latent space into a "semantic" subspace and a "transformational" subspace. This is achieved by using metric learning to encourage representations of semantically similar patches (e.g., different views of the same organelle) to be close in the semantic subspace, while being invariant to geometric and color transformations.
  • Metric Learning: A multi-similarity loss function is employed. This loss encourages representations of an "anchor" patch and its transformed "positive" versions to be close together in the semantic latent subspace, while pushing them away from other unrelated patches.
  • Self-Supervised Training: The entire process is self-supervised, meaning it doesn't require any human-labeled data. The supervision signal comes from the inherent relationships between original and transformed patches.
  • Unsupervised Segmentation: After training, the learned semantic representations of voxels are clustered (using Mini-Batch K-Means) to achieve unsupervised segmentation.

Datasets

The primary dataset used for training and evaluation is a Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) dataset of primary mouse pancreatic islets βcells, specifically the "BetaSeg" dataset [20]. This dataset provides high-resolution, near-isotropic 3D volumes with pre-existing binary segmentation masks for seven categories (centrioles, nucleus, membrane, microtubules, golgi, granules, mitochondria), plus an "unrecognized" category. The authors used a subset of this data, splitting it into training and testing sets for cross-validation. No new benchmarks were introduced; the focus was on applying the method to existing, complex biological data.

Key Insights and Ideas

  • Stereotypical Texture is Key: The paper highlights that while subcellular structures like mitochondria can have highly variable shapes, their local texture and visual features are often stereotypical. This observation is crucial, as it suggests that learning from small patches that capture these local textures is sufficient for distinguishing structures.
  • Semantic vs. Transformational Invariance: The idea of explicitly disentangling a latent space into semantic and transformational components is powerful. By forcing semantic similarity into a dedicated subspace, the model can learn representations that are robust to changes in orientation and scale, which is vital for biological images.
  • Unsupervised Discovery of Structure: The ability of the model to automatically rediscover and separate major cellular components (nuclei, granules, mitochondria) without any prior knowledge is a significant achievement. This demonstrates the potential of self-supervised learning for exploring complex biological data.
  • Dissecting the "Unrecognized": The method not only separates known structures but also provides a finer-grained dissection of the "unrecognized" class, revealing distinct textural patterns that suggest previously uncharacterized substructures or regions.

Key Results

  • Visual Separability: The learned semantic latent subspace (dimensions 1-8) shows clear visual separation of the three major RS classes (nucleus, granules, mitochondria) when visualized with t-SNE, unlike a non-semantic subspace.
  • Transformation Invariance: The semantic subspace demonstrates better invariance to rotations, scaling, and translations compared to other latent subspaces.
  • Accurate Segmentation: The unsupervised segmentation successfully separates nuclei, granules, and mitochondria into distinct clusters.
  • Improved Performance: The proposed method consistently outperforms the CIHS baseline [9] in unsupervised segmentation tasks across various settings, as evidenced by F1-scores (Dice Similarity Coefficient) in cross-validation and testing (e.g., achieving F1-scores around 0.664 for the test set with 2M anchors/stack, compared to CIHS's 0.308 with 10M anchors/stack).
  • Error Analysis: Misclassified regions (e.g., voxels clustered with mitochondria but not) exhibit distinct spatial patterns, providing insights into the model's decision-making and potential areas for refinement.

Limitations and Future Work

  • Texture vs. Shape: The authors acknowledge that the model struggles to differentiate structures with similar textures but different shapes (e.g., mitochondria, Golgi apparatus, and cell membrane).
  • Scale of Representation: The current approach relies on small patches. Future work aims to incorporate larger-scale appearance information to better distinguish structures with similar textures but different shapes.
  • Clustering Robustness: The paper notes that the non-perfect representation might lead to variations in segmentation results depending on the number of clusters (k) chosen for K-Means. Finding an optimal 'k' without supervision remains a challenge.
  • Weakly Supervised Extension: The learned representation is well-suited for weakly supervised segmentation, which could be a promising avenue for future research, requiring only minimal point annotations.
  • Interactive Exploration: The t-SNE visualizations of the latent space offer a powerful tool for interactive exploration, allowing biologists to click on points and see corresponding voxels, facilitating hypothesis generation and refinement.

Summary updated

BibTeX
@inproceedings{han2022self,
  author = {Hongqing Han and Mariia Dmitrieva and Alexander Sauer and Ka Ho Tam and Jens Rittscher},
  title = {{Self-Supervised Voxel-Level Representation Rediscovers Subcellular Structures in Volume Electron Microscopy}},
  booktitle = {Workshops @ CVPR},
  year = {2022},
  doi = {10.1109/cvprw56347.2022.00204},
}
Figure