Abstract
We build a general framework which establishes a one-to-one correspondence between species abundance distribution (SAD) and species accumulation curve (SAC). The appearance rates of the species and the appearance times of individuals of each species are modeled as Poisson processes. The number of species can be finite or infinite. Hill numbers are extended to the framework. We introduce a linear derivative ratio family of models, LDR1, of which the ratio of the first and the second derivatives of the expected SAC is a linear function. A D1/D2 plot is proposed to detect this linear pattern in the data. By extrapolation of the curve in the D1/D2 plot, a species richness estimator that extends Chao1 estimator is introduced. The SAD of LDR1 is the Engen’s extended negative binomial distribution, and the SAC encompasses several popular parametric forms including the power law. Family LDR1 is extended in two ways: LDR2 which allows species with zero detection probability, and RDR1 where the derivative ratio is a rational function. Real data are analyzed to demonstrate the proposed methods. We also consider the scenario where we record only a few leading appearance times of each species. We show how maximum likelihood inference can be performed when only the empirical SAC is observed, and elucidate its advantages over the traditional curve-fitting method.
Introduction
Ecologists and data scientists have long utilized two primary lenses to characterize biodiversity: Species Abundance Distributions (SAD), which describe the relative commonness of species, and Species Accumulation Curves (SAC), which track the discovery of new species as sampling effort increases. Historically, these two metrics have been treated as distinct statistical entities. Furthermore, traditional methods for fitting SACs, such as least-squares curve fitting, are often theoretically flawed because data points on an accumulation curve are inherently dependent—each point contains all the information of the previous points.
This research introduces the Mixed Poisson Partition Process (MPPP), a unified mathematical framework that establishes a rigorous one-to-one correspondence between SAD and SAC. By treating species discovery as a stochastic process, the authors provide new diagnostic tools, a flexible family of models including the power law, and a species richness estimator that improves upon the classic Chao1 method.
The MPPP Framework
The core of the proposed framework is the Mixed Poisson Partition Process. In this model, the appearance of individuals of any given species follows a Poisson process with a specific rate $\lambda$. The process is visualized in Figure 1, which details the generation of species rates and the subsequent arrival of individuals over time, leading to the empirical accumulation curve.
The framework allows for both finite and infinite species counts. A key mathematical insight is that any valid expected SAC must be a Bernstein function, providing a rigorous reason why discovery curves are always non-decreasing and concave. Unlike many traditional models that struggle with "infinite" species counts, the MPPP framework handles them gracefully, showing that the popular power law for species discovery is an extreme case of their model where the total number of species is infinite.
Methodological Innovations
The authors introduce several key tools to improve biodiversity estimation:
- LDR1 (Linear First Derivative Ratio) Family: A new class of models where the ratio of the first and second derivatives of the SAC is a linear function. This family encompasses several famous distributions, including the log-series, the negative binomial, and the power law.
- D1/D2 Diagnostic Plots: A visual tool to check if a dataset fits the LDR1 family. If the plot of the derivative ratios is a straight line, the model is a good fit.
- $E^*(D)$ Estimator: A new species richness estimator that extends the well-known Chao1 estimator. It uses the slope of the D1/D2 plot to predict how many species remain hidden.
- $\rho$-appearance design: A "labor-saving" sampling strategy where researchers stop recording a species once it has been seen a fixed number of times ($\rho$), focusing effort on finding rare species instead.
Key Results and Insights
The framework was validated using four diverse real-world datasets: genomic contig spectra from swine feces, insurance accident data, tomato flower cDNA, and bird abundance data from the North American Breeding Bird Survey.
- Superior Estimation: For datasets with heavy tails (many rare species), the new $E^*(D)$ estimator provided more plausible results than Chao1. In the swine feces and tomato flower data, it correctly identified that species richness was likely infinite or beyond current sampling reach.
- MLE Advantages: The authors demonstrate that using Maximum Likelihood Estimation (MLE) on the actual arrival times of species is statistically superior to fitting a curve to cumulative counts. In simulation studies, the MLE method consistently showed a lower Root Mean Squared Relative Error (RMSRE) compared to traditional curve-fitting methods across various distributions.
- Diagnostic Power: The D1/D2 plots successfully revealed that while some data (like bird abundance) fits simple linear models, others (like the insurance data) require the more complex RDR1 (Rational Derivative Ratio) model.
Limitations and Future Work
While the LDR1 and RDR1 families are flexible, choosing the "right" model still requires careful diagnostic checking. Additionally, while the $\rho$-appearance design saves labor, there is an inherent trade-off in estimation accuracy that requires further optimization. The authors suggest that applying MPPP in a fully non-parametric setting is a promising future direction to avoid the need for model selection entirely.
Summary updated