Unsupervised Learning for Feature Extraction and Temporal Alignment of 3D+t Point Clouds of Zebrafish Embryos

Zhu Chen, Ina Laube, Johannes StegmaierMICCAI 2023 · OpenReview

TL;DR

This paper introduces an unsupervised deep learning method to automatically align the developmental stages of 3D+t zebrafish embryo point cloud data. By learning descriptive features with an autoencoder and then using a regression network for temporal mapping, they achieve high alignment accuracy, significantly reducing manual effort and potential bias in biological research.

What Problem Does This Paper Address?

Understanding the precise developmental stages of organisms like zebrafish is crucial for biomedical research, especially when comparing different individuals or the effects of external factors. Traditionally, this involves manual landmark identification or image-based alignment, which are time-consuming, labor-intensive, and prone to subjective bias. The advent of high-resolution 3D+t microscopy generates complex point cloud data, creating a need for automated methods to synchronize these developmental timelines. This paper tackles the challenge of achieving accurate temporal alignment of these 3D+t point cloud datasets without requiring any manual annotations.

Methods

The core of the proposed method lies in a two-stage deep learning pipeline:

  1. Feature Extraction (Autoencoder): The authors adapt the FoldingNet architecture, originally designed for 3D point cloud processing, to extract descriptive features from each time frame of the zebrafish embryos. Key modifications include:
  2. Using a spherical template instead of a planar one in the decoder for better reconstruction of spherical embryo shapes.
  3. Introducing a Modified Chamfer Distance (MCD) loss function. Unlike the standard Chamfer Distance (which treats all points equally), MCD considers the local density distribution by using point-to-region distances, making it more robust to variations in embryo growth.
  4. Temporal Alignment (Regression Network): The latent feature vectors (1D codewords) extracted by the autoencoder are then fed into a deep regression network. This network is trained to map the features from a "new" embryo to the corresponding time frame indices of a "reference" embryo. This effectively learns a temporal mapping between the two developmental sequences.

A postprocessing step is also included to ensure the generated alignment sequences are monotonically increasing, a necessary condition for temporal order.

Datasets

The study utilizes a dataset published in [4] consisting of four wild-type zebrafish embryos. Each embryo is represented as a 3D point cloud with 370 time frames, captured at one-minute intervals from 4.7 to 10.0 hours post fertilization (hpf).

To overcome the lack of frame-accurate ground truth for temporal alignment, the authors introduce a novel method to synthetically generate validation data. This involves artificially varying the speed of development of a selected embryo (using cosine, sine, Gaussian, and linear stretching/compression) and adding Gaussian-distributed point jitter to create distinct, yet related, sequences.

Key Insights and Ideas

  • Unsupervised Alignment is Feasible: The paper demonstrates that accurate temporal alignment of complex 3D+t biological data can be achieved without manual annotations, opening doors for large-scale, automated analysis.
  • Modified Chamfer Distance is Crucial: The introduction of MCD significantly improves the quality of point cloud reconstruction compared to standard Chamfer Distance, which is vital for learning robust features that capture developmental nuances.
  • Feature Representation Matters: The t-SNE visualizations of the learned feature vectors show a smooth progression with time, indicating that the autoencoder successfully captures the temporal dynamics of embryo development.
  • Spatial Transformations Impact Alignment: While the method is robust to some degree, rotations and centering of point clouds can introduce variance in alignment accuracy, suggesting that spatial context might play a role in developmental stage interpretation.

Key Results

  • High Alignment Accuracy: The method achieves an average temporal mismatch of only 3.83 minutes over an experimental duration of 5.3 hours.
  • Robustness to Shifting Methods: The alignment error is consistently low across different synthetic shifting methods (Cosine, Sine, Gaussian, Faster), with the "Faster" method showing the lowest average mismatch (2.02 minutes).
  • Impact of Spatial Transformations:
    • Scattering + Rotation: Average mismatch of 3.48 minutes.
  • Scattering + Rotation + Centering: Average mismatch of 5.74 minutes.
  • Qualitative Improvement: The modified FoldingNet with MCD shows better reconstruction quality compared to CD, preserving shape and density distribution more accurately.

Limitations and Future Work

  • Lack of Frame-Accurate Ground Truth: While synthetic data generation is a good workaround, the authors acknowledge the need for actual frame-accurate manual labels from multiple raters to better assess real-world performance and inter-rater variability.
  • Impact of Spatial Transformations: The study notes that while the method is robust, rotations and centering can introduce variance. Further work could explore more sophisticated data augmentation or network architectures to improve invariance.
  • Future Directions:
    • Applying the approach to other model organisms and datasets.
  • Conducting extensive studies with human-annotated data.
  • Developing an iteratively optimized spatiotemporal average model to create a reference atlas for precise developmental analysis across experiments.

Summary updated

Abstract

Zebrafish are widely used in biomedical research and developmental stages of their embryos often need to be synchronized for further analysis. We present an unsupervised approach to extract descriptive features from 3D+t point clouds of zebrafish embryos and subsequently use those features to temporally align corresponding developmental stages. An autoencoder architecture is proposed to learn a descriptive representation of the point clouds and we designed a deep regression network for their temporal alignment. We achieve a high alignment accuracy with an average mismatch of only 3.83 min over an experimental duration of 5.3 h. As a fully-unsupervised approach, there is no manual labeling effort required and unlike manual analyses the method easily scales. Besides, the alignment without human annotation of the data also avoids any influence caused by subjective bias.
BibTeX
@misc{chen2023unsupervised,
  author = {Zhu Chen and Ina Laube and Johannes Stegmaier},
  title = {{Unsupervised Learning for Feature Extraction and Temporal Alignment of 3D+t Point Clouds of Zebrafish Embryos}},
  year = {2023},
  eprint = {2502.06543},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.06543},
}
Figure