Multi-Object Portion Tracking in 4D Fluorescence Microscopy Imagery With Deep Feature Maps

Yang Jiao, Mo Weng, Mei YangWorkshops @ CVPR 2019 · DOI

Introduction

Tracking subcellular structures in 4D fluorescence microscopy (3D volumes over time) presents a significant challenge for traditional computer vision algorithms. Unlike rigid macro-objects, protein clusters are morphologically irregular and highly dynamic, frequently undergoing splitting and merging (S&M) events. Standard tracking methods, such as Kalman filters or simple nearest-neighbor linking, often fail when these topological changes occur, leading to fragmented trajectories and inaccurate biological interpretations.

The Deep Feature Map Tracking (DFMT) Framework

To address these challenges, researchers have developed the Deep Feature Map Tracking (DFMT) method. The core innovation of DFMT is twofold: the decomposition of objects into smaller "portions" and the use of high-dimensional deep feature maps for robust matching.

By breaking a complex, deforming object into smaller portions, the tracking problem is simplified from matching entire irregular shapes to finding one-to-one correspondences between stable sub-regions. This approach inherently handles splitting and merging events, as individual portions can be tracked independently across frames. Even if a cluster splits, its constituent portions can be matched to the new fragments in the subsequent time step.

Architecture of the 3D Asymmetric FCDN-103 used for feature extraction, showing the Dense Block (DB) structure and skip connections.

The feature extraction is powered by a 3D Asymmetric Fully Convolutional DenseNet (FCDN-103). Unlike standard 3D networks that use symmetric kernels, this architecture employs asymmetric filters to account for the typical anisotropy in microscopy data. The network generates 64 layers of deep feature maps, providing a rich "fingerprint" for each portion that captures spatial context far beyond raw pixel intensities, making the tracker robust to noise and photobleaching.

Experimental Setup and Data

The DFMT method was validated using challenging real-world biological data: 4D time-lapse imagery of Drosophila (fruit fly) embryos. The target of interest was E-Cadherin, a cell adhesion protein fused with Green Fluorescent Protein (GFP). The dataset consisted of a 60x30x6 micron volume captured every five seconds, creating a dense environment of interacting protein clusters.

Visualization of the 4D microscopy dataset, showing 3D cluster volumes (a.1, a.2) and corresponding 2D projections (b.1, b.2) of E-Cadherin clusters.

Results and Discussion

DFMT demonstrated superior performance compared to existing state-of-the-art trackers, including IOU-based methods and the widely used Fiji/ImageJ "TrackMate" plugin. The system achieved a tracking accuracy of 98.52%.

Most notably, DFMT significantly outperformed traditional methods in identifying split and merge events, reaching an S&M accuracy of 83.87%—a massive 35.48% improvement over traditional methods which typically hover below 50%. The use of Pearson correlation within an extended search volume allowed the tracker to maintain continuity even for fast-moving particles or during temporary signal loss, effectively handling "death" and "birth" events that usually break linear trackers.

Conclusion

The DFMT framework represents a significant advancement in 4D biological imaging. By shifting the focus from whole-object tracking to portion-based matching via deep feature maps, the system provides a robust solution for the chaotic dynamics of the subcellular world. While computationally intensive, the method offers the accuracy required for high-fidelity biological research into protein dynamics and organelle interactions.

Summary updated

BibTeX
@inproceedings{jiao2019multi,
  author = {Yang Jiao and Mo Weng and Mei Yang},
  title = {{Multi-Object Portion Tracking in 4D Fluorescence Microscopy Imagery With Deep Feature Maps}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
  doi = {10.1109/cvprw.2019.00142},
}
Figure