Bayesian inference of fitness landscapes via tree-structured branching processes

Xiang Ge Luo, Jack Kuipers, Kevin Rupp, Koichi Takahashi, Niko Beerenwinkel

TL;DR

This paper introduces FiTree, a novel Bayesian framework that models cancer evolution using tree-structured branching processes. FiTree uniquely integrates tumor phylogenetic trees with population genetics to infer complex, epistatic fitness landscapes from single-cell sequencing data, offering a more accurate and nuanced understanding of cancer progression.

What Problem Does This Paper Address?

Cancer is a complex evolutionary process where cells accumulate mutations, leading to genetic and phenotypic heterogeneity. Understanding this evolution is crucial for developing effective treatments. While single-cell sequencing allows for detailed reconstruction of tumor evolutionary histories (mutation trees), existing models often fail to fully leverage this information. Many approaches either neglect the tree structure, focus only on mutation rates, or oversimplify fitness landscapes by ignoring "epistasis" – the phenomenon where the effect of one mutation depends on the presence of others. This gap limits our ability to accurately predict tumor behavior and identify key drivers of cancer progression.

Methods

FiTree is built upon a tree-structured multi-type branching process. This means it models cell populations as branching processes (where cells divide, mutate, and die) that are organized according to a tree structure representing their evolutionary history.

Key novelties include:

  • Integrating Tree Structure and Branching Processes: Unlike previous methods that treated these separately, FiTree unifies them into a single probabilistic generative model.
  • Epistatic Fitness Parameterization: FiTree uses a fitness matrix to explicitly model both the individual fitness effects of mutations and their pairwise interactions (epistasis). This allows for a more realistic representation of how mutations influence cell growth.
  • Bayesian Inference Scheme: The model employs a Bayesian approach, allowing the incorporation of prior biological knowledge and providing uncertainty estimates for the inferred fitness parameters. This is crucial for understanding the reliability of the findings.
  • Handling Observational Constraints: The model accounts for practical limitations like sampling bias (tumors are more likely to be sampled when larger) and the fact that not all subclones might be observed in the data.

Datasets

The paper uses simulated datasets for benchmarking and a real-world dataset for application:

  • Simulated Datasets: Used extensively to validate the model's approximations and benchmark its performance against existing methods. These simulations involved generating tumor mutation trees under various parameter settings.
  • Single-cell Acute Myeloid Leukemia (AML) Dataset: A dataset from Morita et al. (2020) comprising 123 tumor mutation trees from 31 patients was used to apply FiTree and demonstrate its real-world utility.

Key Insights and Ideas

  • Epistasis is Key: The study strongly suggests that ignoring epistatic interactions leads to an incomplete understanding of cancer evolution. FiTree's ability to capture these interactions provides a more biologically meaningful picture.
  • Tree Structure Matters: Leveraging the evolutionary history encoded in mutation trees is critical for accurate fitness inference, especially when compared to methods that ignore this structure.
  • Uncertainty Quantification is Valuable: The Bayesian framework provides not just point estimates but also credible intervals for fitness parameters, highlighting where our knowledge is more or less certain. This is crucial for interpreting results and making informed decisions.
  • Predictive Power: FiTree's ability to predict future mutational events, with quantified uncertainty, offers a significant step towards personalized medicine and treatment planning.

Key Results

  • Superior Performance in Simulations: FiTree consistently outperformed state-of-the-art methods (SCIFIL, fitclone, diffusion approximation) in recovering fitness landscapes from simulated tumor mutation trees, both for observed and unobserved subclones.
  • Identification of Biological Insights in AML Data: Applying FiTree to AML data revealed epistatic fitness effects consistent with known biological findings, such as the complex interplay between genes like DNMT3A and FLT3.
  • Accurate Approximations: The paper validates the underlying mathematical approximations for subclonal population and sampling time distributions, ensuring the reliability of the model's predictions.
  • Quantified Uncertainty: The 95% highest density intervals for epistatic effects were generally wider than for base effects, indicating greater uncertainty in estimating these complex interactions, which is an important finding in itself.

Limitations and Future Work

  • Computational Complexity: The runtime of FiTree scales with the number of samples, genotypes, and fitness parameters, which can be demanding for very large datasets.
  • Pairwise Epistasis: The current model only considers pairwise epistatic effects. Future work could extend this to higher-order epistasis to capture even more complex interactions.
  • Constant Mutation Rates: FiTree assumes constant mutation rates over time. Real tumors can experience shifts in mutation rates, especially in response to treatment.
  • Incorporating Patient-Specific Factors: The authors suggest integrating patient-specific factors (age, sex, etc.) and treatment effects into the model for more personalized predictions.
  • Handling Tree Uncertainty: While FiTree can handle multiple trees per patient, fully addressing the uncertainty in tree reconstruction itself remains an area for future research.
  • Non-Genetic Events: The framework could potentially be extended to model non-genetic events like epigenetic modifications.

Summary updated

Abstract

Abstract Motivation The complex dynamics of cancer evolution, driven by mutation and selection, underlies the molecular heterogeneity observed in tumors. The evolutionary histories of tumors of different patients can be encoded as mutation trees and reconstructed in high resolution from single-cell sequencing data, offering crucial insights for studying fitness effects of and epistasis among mutations. Existing models, however, either fail to separate mutation and selection or neglect the evolutionary histories encoded by the tumor phylogenetic trees. Results We introduce FiTree, a tree-structured multi-type branching process model with epistatic fitness parameterization and a Bayesian inference scheme to learn fitness landscapes from single-cell tumor mutation trees. Through simulations, we demonstrate that FiTree outperforms state-of-the-art methods in inferring the fitness landscape underlying tumor evolution. Applying FiTree to a single-cell acute myeloid leukemia dataset, we identify epistatic fitness effects consistent with known biological findings and quantify uncertainty in predicting future mutational events. The new model unifies probabilistic graphical models of cancer progression with population genetics, offering a principled framework for understanding tumor evolution and informing therapeutic strategies. Availability and implementation The Python package FiTree and the analysis workflows are available at https://github.com/cbg-ethz/FiTree .
BibTeX
@article{luo2025bayesian,
  author = {Xiang Ge Luo and Jack Kuipers and Kevin Rupp and Koichi Takahashi and Niko Beerenwinkel},
  title = {{Bayesian inference of fitness landscapes via tree-structured branching processes}},
  journal = {Bioinformatics},
  year = {2025},
  doi = {10.1093/bioinformatics/btaf193},
}
Figure