Visualizing and Comparing Machine Learning Predictions to Improve Human-AI Teaming on the Example of Cell Lineage

Jiayi Hong, Ross Maciejewski, Alain Trubuil, Tobias Isenberg

TL;DR

LineageD+ is a new visualization tool that transforms the tedious manual process of mapping plant cell lineages into a collaborative "Human-AI Teaming" experience. By presenting predictions from five different machine learning models through intuitive visual interfaces, it allows biologists to compare AI "opinions," catch errors, and build complex developmental trees with greater efficiency and confidence.

What Problem Does This Paper Address?

In biology, understanding how a single ovum develops into a complex embryo requires mapping "cell lineage"—the history of every cell division. Biologists traditionally do this by looking at 3D microscopy snapshots and working backward in time to identify "sister" cells.

This is a nightmare of a task: it’s incredibly tedious, and the data is "imbalanced" (a cell has dozens of neighbors, but only one true sister). While Machine Learning (ML) can help, a single model often fails because biological data is scarce and noisy. Furthermore, most biologists aren't ML experts; they don't know which model to trust or why a specific prediction was made. This paper addresses the gap between "black-box" AI and the expert intuition required for high-stakes scientific discovery.

Methods

The authors developed LineageD+, a system built on the philosophy of Human-AI Teaming. Instead of trying to find one "perfect" model, they trained an ensemble of five distinct classifiers:

  • Support Vector Machine (SVM)
  • Random Forest
  • K-Nearest Neighbors (KNN)
  • Artificial Neural Network
  • Gaussian Naive Bayes

Key Innovations: * Multi-Model Visualization: The tool uses "semi-donut" charts and bar charts to show how much each model agrees on a potential sister cell. If models disagree, the uncertainty is visually represented as a gray area. * Interactive Weighting: Biologists can adjust "model weights" based on their trust in a specific algorithm for a specific dataset. * Spatial Context: The system links a 2D "icicle plot" (representing the hierarchy) with a 3D view of the embryo, allowing users to see the physical relationship between cells while making lineage decisions. * Error Detection: The tool highlights "similar pairs" using Tanimoto similarity, helping experts quickly spot and correct likely mistakes in the AI's logic.

Datasets

The researchers used a specialized dataset of 93 plant embryo datasets (ranging from 2-cell to 256-cell stages). From these, they extracted 47,132 cell division records characterized by 12 features, such as distance between centers, shared surface area, and cell volume ratios. This dataset is notable for its high imbalance, containing 43,392 negative targets (non-sisters) and only 3,740 positive targets (true sisters).

Key Insights and Ideas

  • The "Ensemble" Advantage: The authors found that while individual models had varying accuracy, in 99.69% of cases, at least one of the five models gave the correct prediction. This justifies the "teaming" approach—the human's job is to pick the winner among the AI's suggestions.
  • Trust through Transparency: Biologists were more willing to use ML when they could see the "disagreement" between models. Visualizing uncertainty didn't make them trust the system less; it made them feel more in control.
  • 3D > 2D Slices: A major "aha" moment for the participants was the transition from traditional 2D microscopy slices to a fully interactive 3D environment. This spatial awareness significantly reduced the cognitive load of identifying cell neighbors.
  • Communicable AI: One participant (P6) described the interaction as "discussing" the lineage with the computer. This suggests that good visualization can turn a tool into a collaborator.

Key Results

  • Usability: The tool earned a System Usability Scale (SUS) score of 77.67, which is considered "good" to "excellent" and a significant improvement over the previous version (LineageD).
  • Efficiency: All six biologists in the study successfully completed the lineage assignments for a 64-cell embryo. They reported that the tool could potentially save massive amounts of time compared to traditional manual methods.
  • Model Performance: In cross-validation, the Random Forest model achieved the highest accuracy (94.24%), while the Bayesian model had the highest recall (84.22%), proving that different models offer different strengths to the "team."

Limitations and Future Work

  • Data Scarcity: With only 93 embryos, the models are still limited. More manually labeled data is needed to improve the underlying AI.
  • Static Models: Currently, the models do not "learn" in real-time from the biologist's corrections. Future work aims to implement Interactive ML, where the models update their parameters as the user provides feedback.
  • Level-by-Level Constraints: The current system predicts one generation at a time. Biologists expressed interest in "sector-by-sector" predictions, which would allow them to focus on one branch of the tree at a time.
  • Generalization: The tool was tested on plant embryos; its effectiveness on other species or different types of biological structures remains to be seen.

Summary updated

Abstract

We visualize the predictions of multiple machine learning models to help biologists as they interactively make decisions about cell lineage-the development of a (plant) embryo from a single ovum cell. Based on a confocal microscopy dataset, traditionally biologists manually constructed the cell lineage, starting from this observation and reasoning backward in time to establish their inheritance. To speed up this tedious process, we make use of machine learning (ML) models trained on a database of manually established cell lineages to assist the biologist in cell assignment. Most biologists, however, are not familiar with ML, nor is it clear to them which model best predicts the embryo's development. We thus have developed a visualization system that is designed to support biologists in exploring and comparing ML models, checking the model predictions, detecting possible ML model mistakes, and deciding on the most likely embryo development. To evaluate our proposed system, we deployed our interface with six biologists in an observational study. Our results show that the visual representations of machine learning are easily understandable, and our tool, LineageD+, could potentially increase biologists' working efficiency and enhance the understanding of embryos.
BibTeX
@article{hong2023visualizing,
  author = {Jiayi Hong and Ross Maciejewski and Alain Trubuil and Tobias Isenberg},
  title = {{Visualizing and Comparing Machine Learning Predictions to Improve Human-AI Teaming on the Example of Cell Lineage}},
  journal = {IEEE Transactions on Visualization and Computer Graphics},
  year = {2023},
  doi = {10.1109/tvcg.2023.3302308},
}

Loading related content...

Figure