Farmers who plant the same crop on the same field year after year often see yields fall and root diseases rise. Soil microbiologists attribute much of this "continuous cropping obstacle" to a collapse of the soil's microbial community: bacteria recede and pathogenic fungi such as Fusarium take over. The evidence comes almost entirely from single-field studies of peanut, tobacco, faba bean, pine plantations and other systems, each run in its own soil and climate. The natural next step is to pool these studies with random-effects meta-analysis, the standard statistical tool for averaging effects across studies whose true effects are allowed to differ. That averaging runs into a specific technical problem. Sequencing reports only proportions, so the analysis cannot tell whether a taxon actually declined or merely shrank as a share of a community that a fungus had flooded. The best existing summary-level method for correcting this assumes that only a few taxa truly change, and that assumption fails in exactly the wholesale collapse soil scientists describe.
We propose to fix the unknown absolute scale with a small number of "anchor" studies, meaning studies that also measured total microbial load, for example by quantitative PCR of bacterial and fungal marker genes. The random-effects structure then carries that scale to every other study through the taxa the studies share. In two sentences: each study's proportion-based effects are off by one unknown per-study constant, and many shared taxa let us estimate the differences between those constants across studies. The single remaining global constant, which current methods pin by assuming sparsity, is pinned instead by the few studies that measured absolute load. Sparsity is no longer needed, and the between-study variance becomes something soil chemistry can explain.
Core Claim
The closest existing work is Melody. It fits a count regression within each study, gives each study a shift parameter $\delta_\ell$ that converts proportion-scale (relative-abundance) coefficients back to the absolute scale, and chooses those shifts by best-subset selection under the assumption that few taxa truly change. Its authors state that "if a covariate causes widespread changes in absolute abundance (e.g., >50% of features change), Melody's performance edge may deteriorate." Melody also treats effects as sparse and fixed. It does not model a between-study variance $\tau^2$ with environmental moderators.
Our claim is that, in a random-effects model, most of what sparsity buys is already identified without it. Only one global scalar per microbial kingdom truly requires outside information, and a handful of anchor studies can supply it. Write $\hat b_{j\ell}$ for study $\ell$'s estimated log fold change of taxon $j$ on the proportion scale (monoculture versus rotation or control), with known sampling variance $s_{j\ell}^2$. The model is
$$\hat b_{j\ell} = \beta_j + x_\ell^\top\gamma_j + \delta_\ell + u_{j\ell} + \varepsilon_{j\ell},\qquad u_{j\ell}\sim N(0,\tau_j^2),\quad \varepsilon_{j\ell}\sim N(0,s_{j\ell}^2).$$
Here $\beta_j$ is the mean absolute log fold change of taxon $j$. The vector $x_\ell$ holds the study's soil moderators (pH, organic carbon, available phosphorus, pesticide history), and $\gamma_j$ gives how each moderator shifts taxon $j$'s effect. The term $\delta_\ell$ is the study's unknown compositional offset, and $u_{j\ell}$ is the genuine between-study deviation, with variance $\tau_j^2$.
Because $\delta_\ell$ is shared by every taxon in a study while the $u_{j\ell}$ average out across hundreds of taxa, the differences $\delta_\ell-\delta_m$ are estimable from shared taxa. What remains unidentified is one additive constant per kingdom. That constant is exactly the component of any effect that is uniform across all taxa, such as a true expansion of total fungal biomass. An anchor study with measured log change in total load $\Lambda_\ell$ adds one constraint:
$$\Lambda_\ell = \log \sum_j \pi_{j\ell}\, \exp\!\left(\hat b_{j\ell}-\delta_\ell\right).$$
Here $\pi_{j\ell}$ is taxon $j$'s baseline proportion in the control soil, which the study's summary statistics can supply. Bacteria (16S sequencing) and fungi (ITS sequencing) are separate compositions with separate offsets. Separate bacterial and fungal load anchors therefore also put the two kingdoms on a common absolute scale. That common scale is what the "bacterial-to-fungal regime shift" question needs and what no proportion-only method can deliver. With more than one anchor the system is overidentified, which gives a built-in falsification test.
Research Program
The work has three parts.
Estimator. We build the estimator by marginal likelihood, alternating between the $\delta_\ell$ (given current $\beta,\gamma,\tau$) and the per-taxon random-effects meta-regressions (given $\delta$). The anchor equations enter as constraints with their own measurement error. Few studies ($K$ small) make $\tau^2$ unstable, as the random-effects literature notes, so we share strength by placing a hierarchical prior on $\tau_j^2$ within a genus or family. This borrows the spirit of the phylogenetically structured shrinkage in BCGLMM. BCGLMM, however, is an individual-level mixed model for disease prediction, whereas ours is a summary-level cross-study synthesis. Within-study summaries come from the same quasi-multinomial regression Melody uses, so the method component is reused and only the between-study layer is new.
Theory. We will characterize identifiability and error. The estimation error in $\delta_\ell-\delta_m$ should shrink roughly as $\bar\tau/\sqrt{J_{\ell m}}$, where $J_{\ell m}$ is the number of shared taxa and $\bar\tau$ is a typical between-study deviation, because the offset is a mean over those many deviations. The global constant's error scales with the anchor measurement error divided by the square root of the number of anchors. We will also state the failure mode plainly. A moderator that uniformly expands all taxa of one kingdom in non-anchor studies can be recovered only through its correlation with anchored studies.
Corpus. We will assemble a summary-statistic corpus of continuous-cropping and soil-degradation studies. The studies in our file seed it:
- peanut fungal succession (Soil Eukaryotic Microorganism Succession... Peanut);
- the pine plantation chronosequence (Pinus sylvestris degradation);
- faba bean wilt (Integrated metabolomics and metagenomics... Fusarium wilt);
- Dictyophora continuous cultivation (Dictyophora degradation);
- Platycodon and peanut intercropping trials (Platycodon intercropping, Peanut intercropping);
- pesticide paddy soils (Paddy subcommunities);
- saline-alkali wheat (Wheat mycobiota);
- areca palm (Areca palm fungal community).
A systematic search expands this seed. Taxa are harmonized at genus level, and at exact sequence variants where raw reads exist, following the exact-sequence approach of the Earth Microbiome Project. The first deliverable is a count of how many studies report absolute load. If fewer than about four anchor studies per kingdom exist, the program changes. It then shifts from real-data claims to a methods contribution plus a recommendation that future trials report load.
The scientific target is a test of the narrative reviews (Evolutions and Managements... Continuous Cropping, Autotoxins in continuous tobacco cropping soils). These reviews assert a universal shift toward fungal dominance. We ask whether the absolute Fusarium expansion and the bacterial decline hold within prediction intervals across soils, or whether pH and phosphorus explain most of $\tau^2$, as the wheat and paddy studies hint.
Baselines
The naive reference point is fixed-effect inverse-variance pooling of per-study centered log-ratio effects, with $\tau^2=0$ and no absolute-scale correction. It shows how much conclusions change once heterogeneity and compositionality are modeled at all. The second reference is standard per-taxon random-effects pooling on the same proportion-scale effects, which adds $\tau^2$ but no offsets. It isolates the compositional correction.
The strongest existing competitor is Melody run on identical summary statistics. It is the only summary-level method in the file that targets absolute-scale effects across studies. Our file quotes Melody as achieving the highest area under the precision-recall curve against MMUPHin, ALDEx2 and ANCOM-BC2 on its synthetic colorectal-cancer benchmark, and a higher Jaccard index in leave-one-study-out tests on five real cohorts. No numeric AUPRC value for Melody was found on Lacuna, so we will rerun its released implementation ourselves rather than quote a figure. We also include a pooled mega-analysis with study as a covariate, the MaAsLin2 design of the gastric mucosal meta-analysis, on the subset of studies with raw reads.
All methods receive the same within-study summaries, and the mega-analysis receives the corresponding raw data. Each method gets its default or recommended tuning, and each gets the same grid search over its own regularization on held-out simulated data. We will count our method as better than Melody under the following conditions:
- In sparse regimes (under 30% of taxa changing), it must match Melody's AUPRC within simulation error.
- In non-sparse regimes (50–80% changing), it must exceed Melody clearly, with confidence intervals on the AUPRC difference excluding zero.
- In all regimes, its 95% intervals for $\beta_j$ must achieve near-nominal coverage.
Two ablations isolate the key ingredient. The first removes the anchors and pins the global constant by a median-zero sparsity rule, which shows how much anchors buy. The second sets $\tau_j^2=0$ while keeping the anchors, which shows how much the random-effects linkage buys.
Why not just…?
Why not just meta-analyze the reported fungal-to-bacterial load ratio as one scalar? For the headline universality question this is a real threat. If enough studies report bacterial and fungal load, a plain scalar random-effects meta-regression on the log ratio answers "is fungal dominance universal?" with no new method, and we will run it first and report it. It cannot say which taxa drive the shift, whether beneficial genera such as Sphingomonas and Lysobacter truly decline in absolute terms (the pattern reported in the faba bean study), or anything about the many studies without load data. Our method's contribution is the taxon-level absolute effects. If those effects add nothing beyond the scalar ratio on held-out studies, we will say so.
Why not just use Melody? Its sparsity rule sets the global constant as if most taxa were unchanged. Under wholesale collapse that constant is biased, and every taxon's absolute effect inherits the bias. Melody's authors anticipate this degradation. Our simulation sweep over the fraction of changing taxa measures exactly where the anchored estimator overtakes it. If the two are indistinguishable even at 80% changing taxa, the anchor idea is unnecessary.
Why not just pool the raw reads and batch-correct, as the gastric study did? That route needs raw FASTQ files and uniform reprocessing, which many agronomic studies do not deposit. More fundamentally, pooling still yields proportions, so it cannot recover absolute change under a fungal bloom any better than summary methods can. Melody's authors report that batch-corrected pooling distorts signals relative to their summary approach. We test this directly on the raw-data subset.
Proposed Experiments
The first experiment is a simulation. We generate semi-synthetic multi-study soil data by resampling baseline compositions from soil samples of the Earth Microbiome Project and applying known absolute effects. The fraction of changing taxa varies from 10% to 80%, $\tau^2$ is tied to simulated pH, the number of studies ranges from 5 to 30, and the number of anchor studies ranges from 0 to 6. Metrics are AUPRC for detecting changed taxa, bias and coverage for $\beta_j$, and recovery of $\gamma_j$.
The second experiment is a sanity check in Melody's home regime. We apply the method to the five colorectal-cancer cohorts and the gut microbiome–metabolome cohorts Melody used, where sparsity plausibly holds, and expect agreement with Melody's signatures. These cohorts lack load anchors, so here our method runs in its no-anchor ablation. Large disagreement would point to a bug or a flaw in the offset estimation.
The decisive real-data check uses the overidentification. In the soil corpus, we hold out each anchor study's measured total load in turn, estimate its offset from shared taxa and the remaining anchors, and predict its load change. If held-out prediction error is no better than predicting zero change, the random-effects linkage is not transferring scale and the central claim fails. If prediction succeeds, we report taxon-level absolute effects with prediction intervals, together with the share of $\tau^2$ explained by pH, phosphorus and pesticide history. That share tells us whether the soil-dysbiosis "law" holds across soils or depends on local soil chemistry.
References
- Melody: meta-analysis of microbiome association studies for discovering generalizable microbial signatures
- Bayesian compositional generalized linear mixed models for disease prediction using microbiome data
- Soil Eukaryotic Microorganism Succession as Affected by Continuous Cropping of Peanut - Pathogenic and Beneficial Fungi were Selected
- Soil Nutrient Dynamics and Fungal Community Shifts Drive the Degradation of Pinus sylvestris var. mongholica Plantations in the Loess Plateau
- Integrated metabolomics and metagenomics uncover pathogenic mechanisms of Fusarium wilt and faba bean defense responses
- Elucidation of Mechanism of Soil Degradation Caused by Continuous Cropping of Dictyophora rubrovalvata Using Metagenomic and Metabolomic Technologies
- Impact of various intercropping modes on soil quality, microbial communities, yield and quality of Platycodon grandiflorum (Jacq.) A. DC.
- Intercropping peanut with traditional Chinese medicinal plants improves soil microcosm environment and peanut production in subtropical China
- Microbial composition, assembly, and functional characteristics of generalized and specialized subcommunities under flooded paddy fields: long-term pesticide versus non-pesticide models
- Relationships between Wheat Development, Soil Properties, and Rhizosphere Mycobiota
- Comparison of Rhizosphere Fungal Community Changes in Healthy and Yellowing-Leaf-Disease-Affected Areca Palms by High-Throughput Sequencing Technology
- A communal catalogue reveals Earth’s multiscale microbial diversity
- Evolutions and Managements of Soil Microbial Community Structure Drove by Continuous Cropping
- Autotoxins in continuous tobacco cropping soils and their management
- Meta-analysis of mucosal microbiota reveals universal microbial signatures and dysbiosis in gastric carcinogenesis