Virtual perturbations to assess explainability of deep-learning based cell fate predictors

Christopher J. Soelistyo, Guillaume Charras, Alan R. LoweWorkshops @ ICCV 2023 · DOI

Deep learning models are often criticized as "black boxes," providing accurate predictions without explaining the underlying biological mechanisms. This paper introduces a "virtual perturbation" framework to probe the internal logic of a cell fate predictor. By systematically manipulating input data and latent representations, the authors demonstrate that their model autonomously identified mechanical crowding—rather than neighbor identity—as the primary driver of apoptosis in competitive cell environments.

The Explainability Gap in Bio-Imaging

Deep learning (DL) has revolutionized bio-imaging by predicting complex outcomes like mitosis or apoptosis. However, the "Rashomon effect" suggests that multiple models can achieve high accuracy using different, potentially non-biological, internal logics. For scientific utility, it is crucial to verify if a model's reasoning aligns with causal biological mechanisms. This study addresses this gap by testing a model's sensitivity to virtual interventions that mimic physical experiments.

Methods: The Virtual Perturbation Framework

The researchers utilized the $\tau$-VAE architecture, a three-stage pipeline consisting of a $eta$-VAE Encoder to compress time-lapse images, a PC Projector to extract interpretable features (e.g., PC0 for cell type, PC1 for nuclear area), and a Temporal Convolutional Network (TCN) to predict cell fate.

The core innovation is Virtual Perturbation, which involves "hacking" the model in two ways:

  • Input Manipulation: Using segmentation masks to identify and "flip" the identity of neighboring cells in raw images to see if the prediction changes.
  • Internal Representation Manipulation: Manually shifting values in the latent space (specifically PC1, which correlates to nuclear compression) to observe the effect on the final prediction.
Schematic of the virtual perturbation framework compared to physical drug treatments, highlighting image and PC manipulation.

Datasets and Biological Context

The study focused on mechanical cell competition between wild-type MDCK cells ($MDCK^{WT}$) and Scribble-knockdown cells ($scrib^{kd}$). In these environments, $scrib^{kd}$ cells are "losers" prone to apoptosis when physically crowded by the "winner" $MDCK^{WT}$ cells. Data consisted of dual-color fluorescence microscopy videos where different cell types were tagged with distinct proteins.

Key Insights

  • Causality over Correlation: Virtual perturbations allowed the authors to confirm that the model learned causal conditions for death, not just correlations.
  • The "Crowding" Factor: The model's PC1 component naturally captured nuclear compression. Manually increasing this "squeeze" in the latent space led to a monotonic increase in apoptosis predictions, mirroring biological reality.
  • Neighbor Identity as a Red Herring: Swapping neighbor identities while maintaining density did not significantly alter predictions. This confirms the model prioritized physical crowding over the specific "species" of neighboring cells.
  • Autonomous Discovery: Without explicit instruction, the model identified compression as a key predictive feature, an insight previously reached through years of manual experimental research.

Key Results

The model showed remarkable robustness to neighbor swapping. The confusion matrices for original and "swapped" data were nearly identical, with mitosis prediction accuracy shifting only slightly from 0.83 to 0.86, indicating that the model is largely indifferent to neighbor identity when predicting fate.

Confusion matrices comparing model performance on original data versus data where neighbor cell types were virtually switched.

Furthermore, the model demonstrated effective Out-of-Distribution (OOD) Detection. When virtual perturbations created physically impossible cell states, the model categorized them as "synthetic" rather than forcing a biological label.

Limitations and Future Work

While the $eta$-VAE achieved significant feature disentanglement, perfect separation of all biological variables remains a challenge. Future work could explore temporal perturbations, such as varying the duration of crowding. The "virtual perturbation" framework itself is highly generalizable, offering a robust method for verifying the "reasoning" of deep learning models across various scientific domains.

Summary updated

BibTeX
@inproceedings{soelistyo2023virtual,
  author = {Christopher J. Soelistyo and Guillaume Charras and Alan R. Lowe},
  title = {{Virtual perturbations to assess explainability of deep-learning based cell fate predictors}},
  booktitle = {Workshops @ ICCV},
  year = {2023},
  doi = {10.1109/iccvw60793.2023.00429},
}
Figure