Statistical Inference Under Phylogenetic Correlation

This direction is split into other directions: phylogenetic confounding. Its last version stays readable and citable here.

Version 1 of 151 papers68 conceptsfrozen 2026-10-04

In standard statistics, we usually assume that our data points are "Independent and Identically Distributed" (i.i.d.). If you measure the height of 100 random people, one person’s height doesn't tell you anything about the next. But in biology, this assumption is almost always false. Because all life shares a common history, species, viruses, and even individual cancer cells are related through a branching tree of ancestry.

This shared history creates a "phylogenetic signal"—a correlation structure where closely related entities are more similar than distant ones simply because they inherited the same traits from a common ancestor. When researchers ignore this structure, they fall into Galton’s Problem: mistaking a shared inheritance for an independent evolutionary response.

The Problem of Spurious Correlation

If you compare two traits across different species—say, brain size and longevity—and find a correlation, you might conclude that big brains cause long life. However, as Emmanuel Paradis and colleagues point out in the documentation for the foundational ape package, this correlation could be a total artifact. If a single ancestor developed both traits, every descendant species will possess them, creating the illusion of a functional relationship across dozens of data points that actually represent only one evolutionary event.

This issue is particularly acute in medical genomics. When studying how viruses like HIV or SARS-CoV-2 evolve to escape the immune system, researchers must distinguish between "functional" mutations (those that actually help the virus survive) and "hitchhiking" mutations (neutral changes that happened to occur in a successful lineage). Vinny Davies and Dirk Husmeier developed the SABRE model specifically to address this, using hierarchical Bayesian priors to filter out the noise of shared ancestry and pinpoint the sparse set of mutations that truly drive vaccine evasion.

Trees as Structural Priors

To fix the i.i.d. violation, researchers are moving away from treating biological data as a flat list. Instead, they embed the phylogenetic tree directly into the statistical framework.

One powerful example is CLIME, developed by Yang Li and Vamsi K. Mootha. Instead of just looking at which genes appear together in different species, CLIME uses a Hidden Markov Model on a tree to identify genes that are gained or lost together across evolutionary time. By accounting for the branching structure, the model avoids "overcounting" ancestral events, leading to much more accurate identification of functional gene modules.

Similarly, in cancer research, Salem Malikić and Niko Beerenwinkel demonstrated with B-SCITE that using a tree to guide the grouping of mutations is far more effective than relying on mutation frequencies alone. The tree acts as a biological filter: if a mutation pattern is physically impossible according to the laws of lineage branching, the model can discard it as technical noise.

Embedding Phylogeny into Deep Learning

The push to account for shared ancestry has recently reached deep learning. Standard Convolutional Neural Networks (CNNs) expect data to have a spatial or temporal order (like pixels in an image). Metagenomic data—the counts of different bacteria in a sample—has no such natural order.

Derek Reiman and Yang Dai solved this with PopPhy-CNN by embedding microbial data into a matrix based on their positions in a phylogenetic tree. This gives the CNN a "spatial" view of evolution, allowing it to learn that a change in one species is biologically similar to a change in its close relative.

In a similar vein, Khalil Ouardini and Michael Jordan introduced TreeVAE, which uses a Variational Autoencoder to reconstruct the states of ancestral cells. By treating single-cell data as a lineage-connected history rather than independent points, the model can "bridge the gap" between observed cells to infer how their progenitors behaved.

Why It Works: Borrowing Strength

The core intuition behind these methods is borrowing strength. If you know that two species are closely related, you can use information from one to help interpret the other.

In microbiome research, Yuhan Chen and Li Ma showed that Bayesian Graphical Compositional Regression outperforms standard methods because it "borrows strength" across related microbial taxa. If several related bacteria all show a slight increase in a diseased patient, the model can aggregate those weak individual signals into a strong, phylogenetically-backed conclusion.

As we move toward larger genomic datasets, the assumption of independence becomes increasingly dangerous. Whether predicting drug resistance in tuberculosis or identifying linguistic universals, the most successful models are those that respect the tree. By formalizing evolutionary history as a statistical dependency, researchers can finally separate the signal of adaptation from the noise of the past.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_statistical_inference_under_phylogenetic_15840f13,
  title = {Statistical Inference Under Phylogenetic Correlation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/statistical-inference-under-phylogenetic-correlation/rdr_15840f137f584f19ae3d5c453d962d5a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_statistical_inference_under_phylogenetic_15840f13_v1,
  title = {Statistical Inference Under Phylogenetic Correlation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/statistical-inference-under-phylogenetic-correlation/rdr_15840f137f584f19ae3d5c453d962d5a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}