Event Detection by Feature Unpredictability in Phase-Contrast Videos of Cell Cultures

Melih Kandemir, José C. Rubio, Ute Schmidt, Christian Wojek, Johannes Welbl, Björn Ommer, Fred A. HamprechtMICCAI 2014 · OpenReview

Abstract

In this work we propose a novel framework for generic event monitoring in live cell culture videos, built on the assumption that unpredictable observations should correspond to biological events. We use a small set of event-free data to train a multioutput multikernel Gaussian process model that operates as an event predictor by performing autoregression on a bank of heterogeneous features extracted from consecutive frames of a video sequence. We show that the prediction error of this model can be used as a probability measure of the presence of relevant events, that can enable users to perform further analysis or monitoring of large-scale non-annotated data. We validate our approach in two phase-contrast sequence data sets containing mitosis and apoptosis events: a new private dataset of human bone cancer (osteosarcoma) cells and a benchmark dataset of stem cells.

Introduction

High-throughput microscopy generates massive amounts of 3D and 4D video data, exceeding the capacity for manual human annotation. While specialized automated tools exist for specific tasks like mitosis detection, they often require extensive hand-labeled training data and lack the flexibility to monitor diverse biological events. This research addresses the need for a generic monitoring tool that identifies significant biological changes—defined as spatially and temporally local visual changes—without requiring event-specific labels.

Methodology: Feature Unpredictability

The proposed framework is based on the concept of Feature Unpredictability. By learning the characteristics of a "normal," stable cell culture, any significant deviation in predictability can be flagged as a biological event.

Overview of the event detection framework based on feature unpredictability.

The process involves several key steps:

  1. Feature Extraction: Video frames are divided into a grid of patches. For each patch, a bank of five standard computer vision features is extracted: Brightness histograms (D-Hist), SIFT descriptors, Local Binary Patterns (LBP), and Histograms of Oriented Gradients (HOG).
  2. The Predictor (MOGP): A Multioutput Multikernel Gaussian Process (MOGP) model performs autoregression, predicting patch features at time $t+1$ based on features at time $t$.
  3. Automatic Feature Selection: A "spike-and-slab" prior allows the model to automatically determine which features are most relevant for prediction, effectively filtering out noise.
  4. Event Mapping: The model is trained on a small set of event-free frames. During inference, the prediction error serves as a proxy for event probability; higher errors indicate a greater likelihood of a biological event.

Experimental Results

The approach was validated on two datasets: human osteosarcoma (U2-OS) cells, where apoptosis was induced via phototoxicity, and a benchmark stem cell dataset (C2C12) with chemically induced apoptosis.

Precision-Recall curves and kernel weights for human osteosarcoma and stem cell datasets.

The MOGP model demonstrated superior performance compared to standard novelty detection methods like One-Class SVMs. Specifically, it achieved an Area Under the Curve (AUC) of 0.77 for osteosarcoma cells and 0.60 for stem cells, compared to 0.57 and 0.53 for the OC-SVM baseline. This performance gain highlights the importance of modeling temporal relationships between frames.

Furthermore, the model's automated feature selection aligned with expert intuition. The model correctly identified that HOG features were most important for detecting mitosis, while LBP and SIFT were more useful for apoptosis—matching the choices human engineers have made for specialized detectors in the past.

Discussion and Conclusion

A key insight of this work is "negative learning"—teaching the system what an event isn't rather than what it is. This allows the model to be sensitive to any departure from the steady state. Different biological processes also leave distinct "error fingerprints" over time; mitosis typically exhibits two sharp error peaks as the cell rounds up and divides, while apoptosis produces a single, broader peak.

While the framework is generic and requires minimal supervision, it currently does not match the precision of fully supervised, task-specific models. Additionally, it requires a baseline of event-free frames for training. Future work will focus on using the specific error signatures to automatically classify the detected events, bridging the gap between detection and categorization.

Summary updated

Abstract

In this work we propose a novel framework for generic event monitoring in live cell culture videos, built on the assumption that unpredictable observations should correspond to biological events. We use a small set of event-free data to train a multioutput multikernel Gaussian process model that operates as an event predictor by performing autoregression on a bank of heterogeneous features extracted from consecutive frames of a video sequence. We show that the prediction error of this model can be used as a probability measure of the presence of relevant events, that can enable users to perform further analysis or monitoring of large-scale non-annotated data. We validate our approach in two phase-contrast sequence data sets containing mitosis and apoptosis events: a new private dataset of human bone cancer (osteosarcoma) cells and a benchmark dataset of stem cells.
BibTeX
@inproceedings{kandemir2014event,
  author = {Melih Kandemir and José C. Rubio and Ute Schmidt and Christian Wojek and Johannes Welbl and Björn Ommer and Fred A. Hamprecht},
  title = {{Event Detection by Feature Unpredictability in Phase-Contrast Videos of Cell Cultures}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2014},
  url = {https://openreview.net/forum?id=3aywriKzKVv2},
}
Figure