Introduction
A 2022 study by Li et al. sent shockwaves through the bioinformatics community by claiming that standard RNA-seq differential expression (DE) tools, such as edgeR and DESeq2, fail to control the False Discovery Rate (FDR) as sample sizes grow into the hundreds. They recommended the non-parametric Wilcoxon rank-sum test as a more robust alternative for large-scale population studies. However, a follow-up study published in Genome Biology by Hejblum et al. identifies a critical flaw in the original simulation design that led to these alarming conclusions.
The Normalization Artifact
The core of the issue lies in the order of operations during data simulation. In RNA-seq, normalization is required to account for differences in "library size" (sequencing depth). The original study generated "null" data—where no biological difference should exist—by permuting raw read counts between samples.
Hejblum et al. argue that permuting raw counts before normalization creates a "library size trap." If a gene's raw count from a sample with a large library is swapped with one from a sample with a small library, the subsequent normalization process creates an artificial outlier. The DE tools were not failing; they were correctly identifying these mathematically induced artifacts as significant differences.
Correcting the Simulation
To test this hypothesis, the authors reproduced the original simulation using GTEx project data (Heart atrial appendage and left ventricle samples) and introduced an "amended" scheme. In this corrected approach, the data is normalized first, and then the values are permuted to create a true null hypothesis that respects the relative expression levels.
They benchmarked several methods, including dearseq (asymptotic and permutation-based), limma-voom, edgeR, DESeq2, NOISeq, and the Wilcoxon rank-sum test, across three scenarios: the original flawed permutation, a "no normalization" scenario, and the corrected "normalization-first" permutation.
Key Results
Under the amended simulation scheme, the "alarming" false positives vanished. Most popular tools, including dearseq, limma-voom, and NOISeq, successfully controlled the FDR even with hundreds of samples.
Furthermore, the study challenges the supposed superiority of the Wilcoxon test. When applied to the same normalized data, the Wilcoxon test also "failed" under the flawed simulation. Under the correct simulation, dearseq (asymptotic) demonstrated higher statistical power than the Wilcoxon test once sample sizes exceeded approximately 20 per group.
Conclusion
This research serves as a vital "correction of the record," reassuring researchers that standard RNA-seq tools remain reliable for large-scale studies. It highlights the critical importance of simulation design: neglecting the mathematical implications of normalization can lead to "garbage in, garbage out" results that misguide the community. Tools like dearseq not only maintain error control but also offer greater flexibility for complex experimental designs compared to simple non-parametric tests.
Summary updated