Automated analysis of high‐content microscopy data with deep learning

Oren Kraus, Ben T. Grys, Jimmy Ba, Yolanda Chong, Brendan J. Frey, Charles Boone, Brenda AndrewsMolecular 2017 · DOI

TL;DR

This paper introduces DeepLoc, a deep learning model for automated analysis of high-content microscopy data, significantly outperforming traditional methods in classifying protein subcellular localization. DeepLoc demonstrates remarkable adaptability, successfully analyzing diverse and challenging datasets with minimal retraining, paving the way for faster and more robust biological discovery.

What Problem Does This Paper Address?

High-content microscopy (HCM) generates vast amounts of image data, crucial for understanding cellular processes by tracking protein localization. However, analyzing this data with traditional machine learning (ML) pipelines is a bottleneck. These methods often require extensive manual tuning and retraining for each new dataset, especially when dealing with variations in imaging conditions, genetic backgrounds, or cell morphologies. This lack of transferability hinders rapid, large-scale biological investigations. The authors aim to overcome these limitations by developing a more robust and adaptable automated analysis tool.

Methods

The core of this work is DeepLoc, a deep convolutional neural network (CNN). Unlike traditional ML approaches that rely on pre-extracted features (e.g., pixel intensity statistics), DeepLoc learns relevant features directly from the raw image data. Specifically, it uses an architecture with 11 layers, including eight convolutional blocks and three fully connected layers, trained on image patches centered on individual yeast cells. This "end-to-end" learning approach allows the network to optimize feature extraction and classification simultaneously.

A key innovation is transfer learning. DeepLoc's pre-trained model can be "fine-tuned" on new datasets with significantly less labeled data than training a model from scratch. This involves initializing the network with the learned weights from a previous task and then retraining only the final layers (or sometimes the entire network with a lower learning rate) on the new data. This makes DeepLoc highly adaptable to different imaging conditions, microscopes, and even novel localization classes.

Datasets

  • Original Training Data: A subset of manually labeled yeast cell images from Chong et al. (2015), comprising ~22,000 images of single cells classified into 15 subcellular compartments.
  • Test Sets:
    • A separate test set of 4,197 single yeast cell images from the Chong et al. (2015) dataset.
  • Images from a pheromone-treated yeast screen (16,596 images) with altered cell morphology.
  • Images from a new genome-wide screen (WT-2017) acquired on a different microscope with different fluorescent markers.
  • Images from a third laboratory (Schuldiner lab) with different imaging protocols and cell clustering.
  • New Benchmarks: The paper implicitly introduces new benchmarks by demonstrating DeepLoc's performance on these diverse and challenging datasets, particularly the pheromone-treated cells and the cross-laboratory dataset, which were difficult for traditional methods.

Key Insights and Ideas

  • Deep Learning for Biological Images: CNNs are not just for general object recognition; they can effectively learn complex biological features directly from microscopy images, surpassing traditional feature-based ML.
  • End-to-End Learning: Learning features and classification simultaneously from raw pixels is more powerful than relying on pre-defined, hand-crafted features, especially for nuanced biological phenotypes.
  • Transfer Learning is Crucial: The ability to fine-tune a pre-trained DeepLoc model on new datasets with minimal labeled data is a game-changer for practical biological research, drastically reducing the effort required for new experiments.
  • Robustness to Cell Morphology: DeepLoc can handle significant changes in cell shape and appearance (e.g., pheromone-treated cells) without explicit retraining, a major advantage over methods sensitive to segmentation.
  • Cross-Laboratory Generalization: DeepLoc's transfer learning capability allows it to analyze data from different labs and microscopes, fostering standardization and data sharing.

Key Results

  • Performance Improvement: DeepLoc achieved a mean average precision of 84% on single-cell classification, a significant improvement over the ensLOC (SVM ensemble) method's 49%. This represents a 71.4% increase in mean average precision.
  • Reduced Training Data: DeepLoc achieved high accuracy with substantially less training data (~70% fewer cells) compared to ensLOC.
  • Adaptability:
    • DeepLoc successfully classified images of pheromone-arrested yeast cells with abnormal morphology without additional training.
  • Using transfer learning, DeepLoc achieved 62.7% accuracy on a new genome-wide screen (WT-2017) with only 5 supplemental training cells per class, a 63.4% improvement over training from scratch.
  • On a third dataset from a different lab, DeepLoc achieved 63.0% accuracy with only 100 samples per class via transfer learning.
  • Protein Dynamics: DeepLoc identified significant localization and abundance changes in proteins in response to mating pheromone, implicating them in conjugation, cell fusion, and polarized growth.

Limitations and Future Work

  • New Localization Classes: While transfer learning is effective, classifying entirely new, punctate localization classes (e.g., cytoplasmic foci, lipid particles) still presents a challenge, with accuracies around 63.8% even when merged with existing classes.
  • Data Availability: The authors emphasize the importance of sharing models and datasets. They provide an open-source implementation of DeepLoc and make their trained models and datasets publicly available to encourage wider adoption.
  • Future Work: The authors suggest that sharing learned feature representations will make models more powerful as they are trained on more varied datasets and classification tasks. They envision this technology aiding many research groups in analyzing their image-based data, accelerating future discoveries.

Summary updated

Abstract

Existing computational pipelines for quantitative analysis of high-content microscopy data rely on traditional machine learning approaches that fail to accurately classify more than a single dataset without substantial tuning and training, requiring extensive analysis. Here, we demonstrate that the application of deep learning to biological image data can overcome the pitfalls associated with conventional machine learning classifiers. Using a deep convolutional neural network (DeepLoc) to analyze yeast cell images, we show improved performance over traditional approaches in the automated classification of protein subcellular localization. We also demonstrate the ability of DeepLoc to classify highly divergent image sets, including images of pheromone-arrested cells with abnormal cellular morphology, as well as images generated in different genetic backgrounds and in different laboratories. We offer an open-source implementation that enables updating DeepLoc on new microscopy datasets. This study highlights deep learning as an important tool for the expedited analysis of high-content microscopy data.
BibTeX
@inproceedings{kraus2017automated,
  author = {Oren Kraus and Ben T. Grys and Jimmy Ba and Yolanda Chong and Brendan J. Frey and Charles Boone and Brenda Andrews},
  title = {{Automated analysis of high‐content microscopy data with deep learning}},
  booktitle = {Molecular},
  year = {2017},
  doi = {10.15252/msb.20177551},
}
Figure