Ecologists measure how "diverse" a community is by counting species and how evenly they are spread. A recent line of work applies the same measures to the genes expressed in a tissue: it treats each gene as a species and each sequencing read as an individual. In human liver, this showed that the gene-expression "ecosystem" loses diversity step by step as alcohol-associated disease worsens, with a few dominant genes taking over. The finding is striking, but it is measured on bulk tissue, meaning a ground-up mixture of millions of cells. That leaves two explanations that were never ruled out. First, the drop could be a sequencing artifact, because diversity scores are very sensitive to how deeply each sample was sequenced and how its library was prepared. Second, it could be a change in which cells are present, with some cell types dying and others moving in, rather than any cell actually expressing a narrower set of genes.
This note proposes to split bulk transcriptome diversity into exactly three parts: a technical part caused by sequencing depth and batch, a mixing part caused by the tissue's cell-type makeup, and a within-cell-type part that measures whether individual cell lineages truly narrow their expression. The split rests on an exact identity from information theory. It becomes measurable once we combine integer-count deconvolution, which estimates each cell type's own gene counts in each sample, with batch harmonization that keeps the data as whole-number counts. The biological question is whether "transcriptomic collapse" in alcohol-associated hepatitis is a hepatocyte defect, a side effect of macrophages moving in, or a depth artifact. The answer is not obvious in advance. Hepatocytes put much of their expression into a few very abundant transcripts, so replacing them with immune cells could actually raise bulk diversity.
Core Claim
Let $p$ be a sample's bulk gene-frequency vector. Let $w_k$ be the fraction of its reads that come from cell type $k$, and let $q_k$ be that cell type's own gene-frequency vector in that sample. Because $p = \sum_k w_k q_k$, bulk Shannon entropy $H$ (the log-scale diversity index used in the liver study) decomposes exactly:
$$H(p) = \sum_k w_k\, H(q_k) \;+\; I(K;G),$$
The first term is the read-weighted average of within-cell-type diversity. The second, $I(K;G)$, is the mutual information between a read's cell of origin $K$ and its gene $G$. That second term is a between-type "mixing" term, bounded above by the entropy of $w$, and it is the gene-expression analogue of ecology's between-community diversity. The claim is that a change in bulk diversity between disease stages can be attributed to three sources. The first is changes in $w$ (composition). The second is changes in each $H(q_k)$ (real within-lineage loss). The third is estimator bias from depth and batch. The plug-in entropy estimator underestimates true entropy by roughly $(S-1)/(2N)$ for $S$ observed genes and $N$ reads, a result that follows from a second-order Taylor expansion. That bias alone can make shallow libraries look "collapsed."
What is new is not the identity itself. It is making each term estimable from real bulk cohorts with honest uncertainty, and then using the result to test a published biological claim that nobody has tried to explain away.
Relation to Prior Work
The motivating result is α-Diversity analysis of hepatic transcriptome reveals distinct pathways in alcohol-associated hepatitis. It computed Shannon, Menhinick, Brillouin, evenness and dominance indices on liver RNA sequencing from 10 healthy controls, 11 steatohepatitis patients and 18 severe hepatitis patients, and found a stepwise diversity drop driven by silenced low-abundance transcripts. A separate marker-based deconvolution in the same study reported a 19% loss of hepatocytes and an 84% rise in Kupffer cells and macrophages. The authors state that they could not tell a hepatocyte defect apart from macrophage takeover, and they did not model batch or depth. We take their cohort and their indices as given, and ask how much of their effect survives the decomposition.
Two lines of work supply our methods. Count Bridges enable Modeling and Deconvolving Transcriptomic Data deconvolves bulk counts into single-cell-level integer profiles that sum exactly to the observed bulk. It does this with a stochastic process defined on whole numbers, and it reports beating CIBERSORTx and MuSiC. This matters because richness and entropy estimators need counts, not continuous embeddings. We reuse it to obtain sample-specific $q_k$. It does not handle cross-study batch effects or compute any diversity measure, and those are the pieces we add. SCDC: Bulk Gene Expression Deconvolution by Multiple Single-Cell RNA Sequencing References deconvolves against several single-cell references and weights them by how well each reconstructs the bulk. We use it as the strongest proportion estimator. Its linear model with fixed reference profiles, however, cannot represent a sample-specific change in within-type diversity, because each $H(q_k)$ is frozen at its reference value.
Batch-correction methods constrain the design more than they supply it. BERMUDA aligns similar clusters across batches with an autoencoder and a distribution-matching loss. BATMAN matches cluster centers globally. Cross-modal transfer learning for mapping bulk transcriptomes at cellular level (POPPY) aligns bulk samples to single-cell atlases in a learned embedding. All three output continuous representations, so none can feed an entropy or richness estimator that needs integer counts. Cross-platform analysis of cancer microarray data improves gene expression based classification of phenotypes is a cautionary result: samples still clustered by laboratory after global normalization. That is why we do not rely on global scaling.
Research Program
The first component is integer-preserving batch harmonization. We model each batch as applying a gene-specific capture efficiency to the true counts. We estimate these efficiencies from genes whose within-cell-type expression should be batch-invariant, such as housekeeping genes inside matched cell types, and optionally from shared control samples. We then harmonize by binomial thinning: each count is randomly downsampled toward the least efficient batch. Thinning keeps the data as whole numbers and preserves their Poisson-like structure. It throws information away, but it never invents reads. That is the right trade-off for richness measures, which are inflated by invented reads. We draw the thinning many times and carry the spread into every downstream estimate.
The second component is per-sample, per-cell-type count estimation. We run Count Bridges on the harmonized bulk counts against a single-cell reference, which yields integer count vectors for each cell type in each sample. The read fractions $w_k$ follow directly from these vectors. We use SCDC proportions as a cross-check on $w_k$.
The third component is depth-standardized diversity and the partition. We compute $H(q_k)$ and the mixing term at a common coverage level, meaning the fraction of reads belonging to already-observed genes. This is the standard ecological fix for comparing communities sampled with unequal effort. We apply bias correction, and we obtain confidence intervals by bootstrapping over reads, thinning draws and deconvolution samples. The output for each sample and stage is a signed attribution of the change in bulk diversity: how much comes from composition, how much from each lineage's own diversity, and how much from depth and batch. We extend the same attribution to the paper's Differential Shannon Diversity, its per-gene entropy contributions, so that the pathway findings (fatty acid oxidation, cholesterol metabolism) can be assigned to a specific lineage.
Baselines
The naive reference point is the published analysis itself: plug-in Shannon, evenness and dominance on raw library-normalized bulk counts, exactly as in the liver study. It defines the effect we try to explain. Beating it means showing, on synthetic data with known truth, that it misattributes changes, and on real data, that its stage effect changes materially under our attribution.
The strongest simple competitor for the technical part is rarefaction: downsample every library to the depth of the shallowest one, then compute diversity. It is the standard ecological answer to unequal depth and is hard to beat on depth alone. We run it with the same number of random draws as our thinning. Our method must match it when only depth differs, and beat it when batches differ in gene-specific capture. The comparison metric is error in the recovered stage difference in Shannon entropy on synthetic data.
The strongest competitor for the composition part is covariate adjustment. We would estimate proportions with SCDC, which reports a Pearson correlation of 0.991 with ground truth on the 6:3:1 three-cell-line mixture and was the only tested method to recover the HbA1c versus beta-cell correlation ($p = 0.0019$) in 77 islet samples. We would then regress bulk Shannon on those proportions and read the residual as "intrinsic" loss. We give this baseline the same references and the same harmonized counts. Beating it means recovering planted within-type diversity changes that a proportion regression either absorbs or hallucinates, measured as correlation and sign accuracy between estimated and true $\Delta H(q_k)$.
We run two ablations. The first removes harmonization and runs deconvolution plus partition on raw counts; it isolates the batch step. The second replaces Count Bridges with SCDC and fixed reference profiles; it isolates the need for sample-specific integer profiles. No reported result found on Lacuna for depth- or batch-corrected transcriptome diversity, or for any cell-type-resolved diversity partition, so there is no prior number for the partition itself.
Why not just…?
Why not just rarefy to equal depth? This is a real threat for the depth component. If the liver cohort's batches differ only in depth, rarefaction may explain as much as our model does, and we will report that plainly. Rarefaction cannot fix gene-specific capture differences between library preparations, and those differences hit exactly the low-abundance transcripts that drove the reported collapse. It also says nothing about composition. The synthetic experiment that varies depth and capture bias separately settles which one matters.
Why not just regress diversity on deconvolved cell proportions? This treats composition as a nuisance covariate with a linear effect. The identity above shows the effect is not linear: the mixing term depends on how different the cell types' profiles are, and not only on their proportions. A residual also cannot say which lineage lost diversity. The planted-collapse simulation tests directly whether regression residuals recover the right lineage and sign.
Why not just apply ComBat or another global correction first? Global location and scale adjustments operate on continuous transformed values. They can produce non-integer or negative "counts," and they shift low counts uniformly. The cross-platform microarray study showed residual laboratory clustering after global normalization, and the low-count genes are the ones diversity indices weight most. We include ComBat-then-plug-in-Shannon as an additional arm in the synthetic test rather than dismissing it.
Why not just sequence single cells and compute diversity per cell type? Where single-cell liver data exist, that is the right validation. We use it as ground truth for $H(q_k)$, although dissociation losses bias it too. It does not help the many existing bulk cohorts, including the ALD cohort, that already carry clinical labels.
Proposed Experiments
The first experiment is synthetic with known truth. Using the 10X PBMC and pancreas data from the BATMAN benchmarks, we build pseudo-bulk samples with controlled composition. In some samples we plant within-type diversity loss by concentrating one cell type's reads onto its top genes. We then layer on depth differences and batch-specific capture bias, applied by thinning. Every baseline and ablation is scored on recovering each partition term. We also check composition accuracy on the SCDC three-cell-line mixture against SCDC's reported 0.991.
The decisive real-data test uses dbGaP phs001807.v1.p1, the ALD cohort. We recompute the Healthy → steatohepatitis → hepatitis diversity trend under the full attribution. We commit in advance to three possible readings. If the hepatocyte within-type term falls with a confidence interval that excludes zero after harmonization, the collapse is a real hepatocyte phenomenon. If the drop is carried by the mixing and depth terms, the published "ecosystem collapse" is largely compositional or technical. If bulk diversity falls while the composition term rises, as hepatocyte dilution would predict, then the within-type loss is larger than bulk suggests.
Two dependencies need flagging. A liver single-cell reference is not listed in the research file and must be sourced. Batch labels or sequencing run metadata for this cohort must also be recovered; if they cannot be, the batch term is limited to depth. As a second real cohort with a known biological anchor, we use the 77 pancreatic islet samples to ask whether beta-cell within-type diversity tracks HbA1c beyond the proportion effect SCDC already found.
References
- α-Diversity analysis of hepatic transcriptome reveals distinct pathways in alcohol-associated hepatitis
- Count Bridges enable Modeling and Deconvolving Transcriptomic Data
- SCDC: Bulk Gene Expression Deconvolution by Multiple Single-Cell RNA Sequencing References
- BERMUDA: a novel deep transfer learning method for single-cell RNA sequencing batch correction reveals hidden high-resolution cellular subtypes
- BATMAN: Fast and Accurate Integration of Single-Cell RNA-Seq Datasets via Minimum-Weight Matching
- Cross-modal transfer learning for mapping bulk transcriptomes at cellular level
- Cross-platform analysis of cancer microarray data improves gene expression based classification of phenotypes