Anisotropic ssTEM Image Segmentation Using Dense Correspondence across Sections

Dmitry Laptev, Alexander Vezhnevets, Sarvesh Dwivedi, Joachim M. BuhmannMICCAI 2012 · DOI

Introduction

In the field of connectomics, researchers aim to map the dense "wiring diagram" of the brain by analyzing Serial Section Transmission Electron Microscopy (ssTEM) data. A significant challenge in this process is the anisotropic nature of the image stacks. While resolution within a single slice (x and y directions) is high, the resolution between slices (z direction) is often much lower, limited by the physical thickness of the tissue cuts.

This discrepancy causes membranes that are not perpendicular to the cutting plane to appear blurred or ambiguous. To resolve these visual ambiguities, this paper introduces a method that incorporates 3D context by aligning neighboring image slices using the "SIFT Flow" algorithm. By pulling in information from adjacent sections, the approach significantly boosts segmentation accuracy over traditional 2D methods.

The Multi-Stage Pipeline

The researchers proposed a multi-stage pipeline that moves beyond simple 2D analysis to leverage the spatial continuity of neural structures.

Overview of the segmentation pipeline: (a) input slices, (b) feature extraction, (c) feature concatenation and Random Forest classification, and (d) Graph Cut post-processing.
  1. Dense Correspondence (SIFT Flow): To handle shifts and deformations between slices caused by the cutting process, the SIFT Flow algorithm finds a non-linear warping that maps pixels from neighboring slices ($k-1$ and $k+1$) to the current slice ($k$).
  2. Feature Extraction: For every pixel, 626 features (e.g., Gaussian blurs, Sobel filters) are calculated. By using the SIFT Flow alignment, features from the current pixel are concatenated with features from corresponding pixels in neighboring slices, creating an 1,878-dimensional feature vector.
  3. Random Forest Classification: This extended vector is fed into a Random Forest classifier to predict the probability of each pixel being a membrane.
  4. Graph Cut Segmentation: A final Graph Cut step ensures spatial consistency and incorporates geometric constraints like "gradient flux" to maintain biologically plausible neural structures.

Leveraging 3D Context via SIFT Flow

A key insight of this work is that warping is mandatory. Simply looking at the same (x, y) coordinate in the next slice often introduces noise because the tissue shifts during preparation. SIFT Flow treats alignment as an "optical flow" problem, finding the optimal non-linear transformation to align structures across the z-axis.

Visualizing dense correspondence: $I^1$ and $I^2$ are adjacent slices. $F_{1,2}(I^1)$ shows the first slice warped to match the second. The difference map and grid illustrate the non-linear deformation applied.

By mathematically warping clarity from a sharp neighbor into a blurred region of the current slice, the classifier gains the evidence needed to make confident predictions. The authors found that using warped context improved pixel error by 6.4% compared to a single-slice approach, whereas using "direct" (unwarped) context actually degraded performance in some metrics.

Performance and Practicality

The study utilized the ISBI 2012 challenge dataset (Drosophila larva ventral nerve cord). The proposed "Dense ETH" method achieved competitive results:

  • Accuracy: The method outperformed several other automated approaches and stayed within striking distance of computationally expensive deep learning models.
  • Efficiency: Unlike deep neural networks of the time, which could take a week to train, the Random Forest approach requires only hours and less specialized hardware, making it highly practical for standard laboratory environments.
  • Human Gap: While automated performance is high, a gap remains; human annotators achieved a pixel error of $6.7 \times 10^{-2}$ compared to the paper's $7.9 \times 10^{-2}$.

Conclusion

By integrating dense correspondence into the segmentation pipeline, this method effectively addresses the challenges of anisotropy in ssTEM data. Future work may involve expanding the context window beyond immediate neighbors or implementing more sophisticated 3D post-processing to ensure membranes form closed, consistent tubes across entire image stacks, further closing the gap with human performance.

Summary updated

BibTeX
@inproceedings{laptev2012anisotropic,
  author = {Dmitry Laptev and Alexander Vezhnevets and Sarvesh Dwivedi and Joachim M. Buhmann},
  title = {{Anisotropic ssTEM Image Segmentation Using Dense Correspondence across Sections}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2012},
  doi = {10.1007/978-3-642-33415-3\_40},
}

Loading related content...

Figure