Correlations and Co-Occurrences of Taxa: the Role of Temporal, Geographic and Taxonomic Restrictions

Aleksi Kallio, Kai Puolamäki, Mikael Fortelius, Heikki MannilaPalaeontologia 2011

Abstract

Correlation between occurrences of taxa is a fundamental concept in the analysis of presence-absence data. Such correlations can result from ecologically relevant processes, such as existence and evolution of species communities. Correlations are typically quantified by some sort of similarity index based on co-occurrence counts. We argue that the individual values of a similarity index are not useful as such: rather, we have to be able to estimate the statistical significance of the index value. Secondly, we argue that before computing the correlations one has to carefully select what is the underlying base set of locations for which the co-occurrence counts, similarity indices, and their significance is computed. We demonstrate base set selection with synthetic examples and conclude with an analysis of real data from a large database of fossil land mammals.

Introduction

In ecology and paleontology, determining whether two species co-occur more or less often than expected by chance is a foundational task. Traditionally, researchers have relied on similarity indices like Jaccard or Dice to quantify these relationships. However, these methods often fail to distinguish between genuine ecological interactions and trivial artifacts of geography or time. This paper proposes a more rigorous statistical framework centered on significance testing and the careful selection of the "base set" of data.

The Pitfalls of Similarity Indices

Similarity indices provide a numerical score of co-occurrence but lack a measure of statistical significance. Furthermore, they often ignore "double absences"—locations where neither species is present. In a global dataset, two species endemic to the same continent will appear highly correlated simply because they are both absent everywhere else. This "trivial" correlation masks the actual interactions researchers seek to study, such as competitive exclusion or shared micro-habitats.

Contingency table demonstrating independent distributions with high similarity scores.

As shown in the table above, species can be distributed completely independently yet yield a high similarity score. In this example, the Jaccard index is approximately 0.82, yet the probability of species A occurring is 0.9 regardless of species B's presence. Without significance testing, such independent distributions are easily misinterpreted as meaningful correlations.

Methodology: Significance and Base Sets

To address these flaws, the authors advocate for a two-pronged approach:

  1. Fisher’s Exact Test (mid-P variant): This calculates formal p-values for correlations by comparing observed co-occurrences against a null hypothesis of independence. To manage the high volume of pairwise comparisons, the Benjamini-Hochberg method is used to control the False Discovery Rate (FDR) at 5%.
  2. Base Set Selection: This involves restricting the background "universe" of locations to control for confounding variables. Filters can be applied based on geography (restricting to overlapping ranges), time (restricting to overlapping lifetimes), or taxonomy (restricting to sites where related families are present to mitigate sampling bias).

Geographic Base Set Selection

The choice of base set acts as a controlled experiment. If a global base set is used to compare two African species, the math will almost always yield a positive correlation due to their shared absence in other regions.

Synthetic global distribution of two species used to test base set restrictions.

By restricting the base set to the union or intersection of the species' ranges (indicated by the dashed boxes in the figure), researchers can force the analysis to ignore broad regional effects. In synthetic tests, shifting from a global base set to a restricted African base set flipped a non-significant positive correlation into a statistically significant negative correlation, completely changing the ecological interpretation.

Empirical Results: The NOW Database

The methodology was applied to the Neogene of the Old World (NOW) database, focusing on Eurasian fossil land mammals from the Miocene to Pleistocene. The analysis of 217 sites and 169 genera revealed the critical importance of temporal filtering. Without temporal restrictions, the dataset yielded 1,645 significant correlations—most of which were trivial results of species living in different eras. Applying a temporal filter reduced this to just 49 genuine correlations.

These remaining correlations successfully mapped to known ecological realities. The algorithm clustered forest-adapted taxa (e.g., Tapirus, Cervus) separately from open-woodland and glacial taxa (e.g., Mammuthus, Equus), demonstrating that the method can effectively extract biological signals from the fossil record.

Conclusion

To find ecologically meaningful correlations in presence-absence data, researchers must move beyond simple similarity indices. By combining Fisher’s Exact Test with rigorous base set restrictions, it is possible to filter out biogeographical and temporal noise, revealing the underlying dynamics of species communities. While these restrictions reduce statistical power, they ensure that the resulting correlations represent genuine biological relationships rather than mathematical artifacts.

Summary updated

Abstract

Correlation between occurrences of taxa is a fundamental concept in the analysis of presence-absence data. Such correlations can result from ecologically relevant processes, such as existence and evolution of species communities. Correlations are typically quantified by some sort of similarity index based on co-occurrence counts. We argue that the individual values of a similarity index are not useful as such: rather, we have to be able to estimate the statistical significance of the index value. Secondly, we argue that before computing the correlations one has to carefully select what is the underlying base set of locations for which the co-occurrence counts, similarity indices, and their significance is computed. We demonstrate base set selection with synthetic examples and conclude with an analysis of real data from a large database of fossil land mammals.
BibTeX
@inproceedings{kallio2011correlations,
  author = {Aleksi Kallio and Kai Puolamäki and Mikael Fortelius and Heikki Mannila},
  title = {{Correlations and Co-Occurrences of Taxa: the Role of Temporal, Geographic and Taxonomic Restrictions}},
  booktitle = {Palaeontologia},
  year = {2011},
}
Figure