Mean Shift Clustering as a Loss Function for Accurate and Segmentation-aware Localization of Macromolecules in Cryo-electron Tomography

Lorenz Lamm, Ricardo D. Righetto, Tingying Peng

TL;DR

This paper introduces a novel "Mean Shift loss" function that enables end-to-end training of deep learning models for precisely locating proteins in cryo-electron tomography (Cryo-ET) images. By integrating a differentiable Mean Shift clustering directly into the training process, the method achieves more accurate protein center identification, especially for non-spherical molecules, and handles sparse annotations more effectively than previous approaches.

What Problem Does This Paper Address?

Cryo-electron tomography (Cryo-ET) is a powerful technique for visualizing cellular structures in their native state. A crucial step for analyzing these structures, particularly for techniques like subtomogram averaging (STA), is accurately locating individual protein molecules within the tomograms. Existing methods often involve a two-step process: first, training a neural network to segment protein shapes, and then using a separate Mean Shift clustering algorithm as post-processing to find the protein centers. This separation can lead to inaccuracies and inconsistencies in the final localization, especially for proteins with complex or non-spherical shapes. Furthermore, the generation of detailed segmentation masks for training can be labor-intensive, and existing datasets often suffer from incomplete annotations.

Methods

The core innovation is the development of a differentiable, scoremap-guided Mean Shift clustering module that is integrated directly into the neural network's loss function. This allows for end-to-end training, optimizing protein localization directly.

Key aspects of their method include:

  • Scoremap Guidance: Instead of relying solely on raw voxel data, the Mean Shift clustering is guided by the confidence scores predicted by the U-Net segmentation network.
  • Point Sampling: To make the computationally intensive Mean Shift clustering feasible during training, the authors randomly sample a limited number of points around the ground truth protein locations. This strategy also helps to focus training on regions with annotations and makes the process more robust to sparse ground truth data.
  • Differentiable Implementation: The Mean Shift clustering is implemented in a way that allows gradients to flow back through it, enabling the entire network to be trained using backpropagation.
  • Combined Loss: The Mean Shift loss can be used either alone or in combination with traditional segmentation losses (like Dice loss), allowing for a balance between accurate shape preservation and precise localization.

Datasets

The authors evaluated their method on several datasets:

  • Sandclock: A synthetic toy dataset created by the authors, featuring two spheres to demonstrate basic functionality.
  • Shrec Challenge Cryo-ET dataset: A synthetic dataset designed to simulate realistic tomograms with proteins of varying sizes.
  • Ribo dataset: An experimental dataset containing tomograms of ribosomes, providing a real-world benchmark.
  • Ribo (sparse) dataset: A variation of the Ribo dataset where training was performed with only a single ground truth position and mask per patch, specifically to test the method's robustness to sparse annotations.

Key Insights and Ideas

  • End-to-End Localization: Directly optimizing for protein center positions within the loss function is more effective than a post-processing approach.
  • Scoremap Importance: The confidence scores from the segmentation network are valuable for guiding the clustering process towards accurate centers.
  • Sparse Annotation Robustness: The point sampling strategy makes the method surprisingly resilient to incomplete ground truth annotations, a common issue in Cryo-ET.
  • Shape Matters: The Mean Shift loss shows a particular advantage for localizing non-spherical proteins, where segmentation-based approaches can struggle to pinpoint the exact center.

Key Results

  • Improved Localization Accuracy: The Mean Shift loss consistently led to more accurate cluster center positions compared to using only Dice loss, as evidenced by lower distance metrics (distpred, distGT) and higher F1-scores, especially for non-globular protein shapes.
  • Enhanced F1-Scores: For the Shrec dataset, the Mean Shift loss alone achieved the highest F1-scores, indicating better overall localization accuracy even with potential outliers.
  • Effective with Sparse Data: In the Ribo (sparse) experiment, the combined loss function maintained similar performance to full annotation training, while the Dice loss alone showed a significant drop in accuracy.
  • More Precise Score Maps: The Mean Shift loss generates score maps that are more tailored for precise localization, leading to better clustering outcomes.

Limitations and Future Work

  • Outlier Sensitivity: While generally robust, the Mean Shift loss can sometimes produce significantly deviant outlier cluster centers, impacting distance metrics (as seen with high standard deviations in the Shrec results).
  • Multi-Class Extension: The authors suggest extending the loss function to handle multiple classes of proteins.
  • Downstream Task Evaluation: Further evaluation on how improved protein localization impacts downstream tasks like subtomogram averaging is proposed.
  • More Diverse Datasets: Testing on a wider range of experimental datasets with more diverse and non-globular protein shapes is also recommended.

Summary updated

Abstract

ABSTRACT Cryo-electron tomography allows us to visualize and analyze the native cellular environment on a molecular level in 3D. To reliably study structures and interactions of proteins, they need to be accurately localized. Recent detection methods train a segmentation network and use post-processing to determine protein locations, often leading to inaccurate and inconsistent locations. We present an end-to-end learning approach for more accurate protein center identification by introducing a differentiable, scoremap-guided Mean Shift clustering implementation. To make training computationally feasible, we sample random cluster points instead of processing the entire image. We show that our Mean Shift loss leads to more accurate cluster center positions compared to the classical Dice loss. When combining these loss functions, we can enhance 3D protein shape preservation and improve clustering with more accurate, localization-focused score maps. In addition to improved protein localization, our method provides more efficient training with sparse ground truth annotations, due to our point sampling strategy.
BibTeX
@inproceedings{lamm2024mean,
  author = {Lorenz Lamm and Ricardo D. Righetto and Tingying Peng},
  title = {{Mean Shift Clustering as a Loss Function for Accurate and Segmentation-aware Localization of Macromolecules in Cryo-electron Tomography}},
  booktitle = {ISBI},
  year = {2024},
  url = {https://openreview.net/forum?id=g085FRjZI3},
}
Figure