Smooth 2D manifold extraction from 3D image stack

Asm Shihavuddin, Sreetama Basu, Elton Rexhepaj, Felipe Delestro, Nikita Menezes, Séverine M. Sigoillot, Elaine Del Nery, Fekrije Selimi, Nathalie Spassky, Auguste GenovesioNature Communications 2017 · DOI

Abstract

Three-dimensional fluorescence microscopy followed by image processing is routinely used to study biological objects at various scales such as cells and tissue. However, maximum intensity projection, the most broadly used rendering tool, extracts a discontinuous layer of voxels, obliviously creating important artifacts and possibly misleading interpretation. Here we propose smooth manifold extraction, an algorithm that produces a continuous focused 2D extraction from a 3D volume, hence preserving local spatial relationships. We demonstrate the usefulness of our approach by applying it to various biological applications using confocal and wide-field microscopy 3D image stacks. We provide a parameter-free ImageJ/Fiji plugin that allows 2D visualization and interpretation of 3D image stacks with maximum accuracy.

Introduction

In biological research, many samples are effectively "2.5D"—such as flat cell layers or curved membranes. Because these structures are rarely perfectly planar, a single 2D snapshot cannot capture the entire structure in focus. Researchers typically acquire a "stack" of images at varying depths (a 3D volume) and then apply Maximum Intensity Projection (MIP) to flatten the data into a single image.

However, MIP is "oblivious" to spatial context. It simply selects the brightest voxel at every (x, y) coordinate. This often leads to the "teleportation" of out-of-focus artifacts, such as bright debris or distant blood vessels, into the final projection. Such discontinuous extractions can lead to erroneous biological conclusions, such as the false appearance of contact between distant organelles. To address this, we introduce Smooth Manifold Extraction (SME), a parameter-free algorithm that transforms complex 3D microscopy image stacks into accurate, continuous 2D projections while preserving local spatial relationships.

Methodology

SME treats the extraction of a 2D surface as a global optimization problem rather than a series of independent pixel choices. The algorithm follows a four-step process:

  1. Focus Measurement: The algorithm determines local focus. For confocal images, raw intensity is used; for wide-field images, a "Sum of Modified Laplacian" identifies sharp edges.
  2. Classification via K-means: By analyzing the "z-profile" (the column of pixels at each coordinate), a 3-class k-means clustering on power spectra categorizes pixels as Foreground, Background, or Uncertain.
  3. Cost Function Optimization: SME calculates an optimal "index map" (a 2D surface $Z$) by minimizing a cost function that balances data attachment (staying near in-focus foreground voxels) with smoothness (preventing abrupt jumps between distant layers).
  4. Propagation: The algorithm allows the smooth manifold found in foreground areas to propagate through noisy or empty background regions, effectively "ignoring" out-of-focus noise.

The tool is available as a parameter-free plugin for ImageJ/Fiji, ensuring accessibility for the broader biological community.

Results and Key Insights

The authors evaluated SME using synthetic benchmarks and diverse biological samples, including tubulin networks and ependymal cells.

Comparison of Maximum Intensity Projection (MIP) and Smooth Manifold Extraction (SME) for widefield tubulin (a, c) and confocal ependymal cell (b, d) images.

Quantitative and Qualitative Superiority

On synthetic data, SME consistently outperformed MIP and other methods like Extended Depth of Field (EDF), showing lower Root Mean Square Error (RMSE) and higher Signal-to-Noise Ratio (SNR). Qualitatively, SME significantly improves contrast in structures like tubulin filaments, resolving individual strands that appear blurred in MIP (see panels a and c in the figure above).

Biological Accuracy and Spatial Consistency

A major advantage of SME is its ability to maintain spatial coherence. In ependymal cell samples, MIP often mistakenly includes blood vessels located deep below the cell layer of interest. SME successfully excludes these artifacts by maintaining the manifold on the apical cell surface (see panels b and d in the figure above). Furthermore, SME facilitates accurate multi-channel alignment. By using a single "reference" channel to define the manifold, researchers ensure that co-localization signals originate from the same 3D voxel, preventing false positives.

Conclusion and Future Directions

While SME provides a significant leap in 2D projection accuracy, it is designed for "2.5D" samples that can be represented as a single continuous sheet. For truly complex 3D objects with multiple overlapping layers of interest, interactive 3D rendering remains the gold standard. However, for the majority of biological applications currently relying on MIP, SME offers a robust, parameter-free alternative that preserves biological truth by enforcing spatial smoothness.

Summary updated

Abstract

Three-dimensional fluorescence microscopy followed by image processing is routinely used to study biological objects at various scales such as cells and tissue. However, maximum intensity projection, the most broadly used rendering tool, extracts a discontinuous layer of voxels, obliviously creating important artifacts and possibly misleading interpretation. Here we propose smooth manifold extraction, an algorithm that produces a continuous focused 2D extraction from a 3D volume, hence preserving local spatial relationships. We demonstrate the usefulness of our approach by applying it to various biological applications using confocal and wide-field microscopy 3D image stacks. We provide a parameter-free ImageJ/Fiji plugin that allows 2D visualization and interpretation of 3D image stacks with maximum accuracy.
BibTeX
@inproceedings{shihavuddin2017smooth,
  author = {ASM Shihavuddin and Sreetama Basu and Elton Rexhepaj and Felipe Delestro and Nikita Menezes and Séverine M. Sigoillot and Elaine Del Nery and Fekrije Selimi and Nathalie Spassky and Auguste Genovesio},
  title = {{Smooth 2D manifold extraction from 3D image stack}},
  booktitle = {Nature Communications},
  year = {2017},
  doi = {10.1038/ncomms15554},
}
Figure