Impact of human gene annotations on RNA-seq differential expression analysis

Yu Hamaguchi, Chao Zeng, Michiaki Hamada

Abstract

Differential expression (DE) analysis of RNA-seq data typically depends on gene annotations. Different sets of gene annotations are available for the human genome and are continually updated–a process complicated with the development and application of high-throughput sequencing technologies. However, the impact of the complexity of gene annotations on DE analysis remains unclear. Using “mappability”, a metric of the complexity of gene annotation, we compared three distinct human gene annotations, GENCODE, RefSeq, and NONCODE, and evaluated how mappability affected DE analysis. We found that mappability was significantly different among the human gene annotations. We also found that increasing mappability improved the performance of DE analysis, and the impact of mappability mainly evident in the quantification step and propagated downstream of DE analysis systematically. Our findings indicate that the growth and complexity of gene annotations negatively impact the performance of DE analysis, suggesting that an approach that excludes unnecessary gene models from gene annotations improves the performance of DE analysis.

Introduction

In transcriptomics, RNA-seq is the standard method for identifying changes in gene activity. This process relies on reference gene annotations—genomic maps that define the boundaries of genes and their various isoforms. As sequencing technologies advance, these maps (such as GENCODE and RefSeq) have become increasingly dense, incorporating a vast array of non-coding RNAs and complex splice variants. While comprehensive, this complexity introduces "mapping uncertainty," where individual RNA fragments cannot be uniquely assigned to a single gene model. This study investigates how this "crowded room" effect impacts the accuracy of differential expression (DE) analysis.

Methodology: Defining Mappability

To quantify annotation complexity, the researchers introduced the metric of mappability. Mappability is defined as the fraction of reads derived from a transcript that can be uniquely and correctly aligned back to that transcript. A value of 1.0 represents a clear, unambiguous signal, while lower values indicate high uncertainty.

The study benchmarked four major analysis pipelines:

  1. TopHat2–Cufflinks
  2. HISAT2–StringTie–Ballgown
  3. STAR–RSEM–EBSeq
  4. Kallisto–Sleuth

These were tested using both simulated data (generated via the polyester R package) and experimental data from the Microarray Quality Control (MAQC) project, validated against TaqMan qRT-PCR measurements.

Results

Annotation Complexity and Performance

The analysis revealed that different databases possess distinct "personalities" regarding mappability. NONCODE, which focuses on long non-coding RNAs, showed higher average mappability due to simpler gene models. Conversely, RefSeq and GENCODE exhibited lower mappability, largely driven by a high volume of automated and overlapping isoforms.

Figure 1: Comparison of mappability across annotations and its correlation with DE analysis accuracy.

As shown in Figure 1, there is a direct, monotonic relationship between transcript mappability and the Area Under the Curve (AUC) for DE analysis. As mappability increases, the performance of nearly all pipelines improves. The study identified the quantification step—where reads are assigned to specific isoforms—as the primary bottleneck where complexity-induced errors occur. These errors then propagate downstream, leading to an increase in false positives.

The Benefit of Tailored Annotations

A key finding of the study is that "less is more." By creating "tailored" annotations—pruning the gene map to remove transcripts not expressed in the specific samples being studied—the researchers were able to reduce noise. This tailoring approach improved AUC scores by an average of 0.005 points across all tested pipelines, demonstrating that removing unnecessary gene models enhances the reliability of the results.

Discussion and Conclusion

The results suggest that the continuous expansion of gene annotations, while intended to be comprehensive, can paradoxically hinder the precision of RNA-seq analysis. The STAR-RSEM-EBSeq pipeline was noted as being particularly sensitive to these effects.

Future genomic research may benefit from dynamic, automated methods to tailor gene annotations to specific tissues or experimental conditions. By reducing the complexity of the reference map to match the biological reality of the sample, researchers can achieve more accurate and reproducible differential expression results.

Summary updated

Abstract

Abstract Background: Differential expression (DE) analysis of RNA-seq data typically depends on gene annotations. Different sets of gene annotations are available for the human genome and are continually updated–a process complicated with the development and application of high-throughput sequencing technologies. However, the impact of the complexity of gene annotations on DE analysis remains unclear. Results: Using “mappability”, a metric of the complexity of gene annotation, we compared three distinct human gene annotations, GENCODE, RefSeq, and NONCODE, and evaluated how mappability affected DE analysis. We found that mappability was significantly different among the human gene annotations. We also found that increasing mappability improved the performance of DE analysis, and the impact of mappability mainly evident in the quantification step and propagated downstream of DE analysis systematically. Conclusions: We assessed how the complexity of gene annotations affects DE analysis using mappability. Our findings indicate that the growth and complexity of gene annotations negatively impact the performance of DE analysis, suggesting that an approach that excludes unnecessary gene models from gene annotations improves the performance of DE analysis.
BibTeX
@inproceedings{hamaguchi2021impact,
  author = {Yu Hamaguchi and Chao Zeng and Michiaki Hamada},
  title = {{Impact of human gene annotations on RNA-seq differential expression analysis}},
  booktitle = {BMC},
  year = {2021},
  doi = {10.1186/s12864-021-08038-7},
}
Figure