Impact of gene annotation choice on the quantification of RNA-seq data

David Chisanga, Yang Liao, Wei ShiBMC Bioinformatics 2022 · DOI

TL;DR

This study investigates how different gene annotation databases (Ensembl and RefSeq) impact the accuracy of RNA sequencing (RNA-seq) gene expression quantification. The authors found that the more conservative RefSeq annotation generally leads to more accurate results, surprisingly, even more so than the recently updated RefSeq versions which incorporate sequencing data.

What Problem Does This Paper Address?

RNA sequencing is a powerful tool for understanding gene expression, but its accuracy heavily relies on the quality of gene annotations. These annotations, essentially maps of genes on a genome, are crucial for counting RNA fragments that correspond to specific genes. Different annotation databases exist (like Ensembl and RefSeq), and their comprehensiveness and curation methods vary. While it's understood that annotations influence RNA-seq results, there's a lack of clear understanding on how these differences, especially with newer, rapidly expanding annotations, affect quantification accuracy. This paper aims to bridge this gap by systematically comparing the impact of prominent human gene annotations on RNA-seq quantification accuracy, using established ground truths.

Methods

The researchers employed a standard RNA-seq analysis pipeline, but with a focus on comparing the output when using different gene annotations. Key steps included:

  • Read Mapping: RNA-seq reads were mapped to the human reference genome (GRCh38) using the Rsubread aligner. The align function was configured with specific parameters for fragment alignment and mismatch tolerance.
  • Gene Counting: The featureCounts program was used to count mapped fragments (read pairs) assigned to genes based on exon overlap.
  • Normalization: Gene-level read counts were transformed using the voom function and then normalized using library size, quantile, and TMM methods. The transformed data was converted to log2FPKM for analysis.
  • Annotation Comparison: Three human gene annotations were used: a recent Ensembl annotation, a recent RefSeq annotation (RefSeq-NCBI), and an older RefSeq annotation (RefSeq-Rsubread).
  • Validation: The accuracy of quantification from each annotation was assessed using three independent ground truths:
  • Genome-wide titration monotonicity built into the SEQC dataset.
  • TaqMan real-time PCR (RT-PCR) data.
  • Illumina BeadChip microarray data.
  • Expression Filtering: Genes with low or no expression were filtered out to assess the impact of annotation on commonly analyzed datasets.

Datasets

The primary dataset used was the SEQC/MAQC III consortium benchmark RNA-seq dataset. This dataset includes:

  • Universal Human Reference RNA (UHRR) - Sample A
  • Human Brain Reference RNA (HBRR) - Sample B
  • Mixture samples: 75%A/25%B (Sample C) and 25%A/75%B (Sample D)

These samples were sequenced in four replicate paired-end libraries.

For validation, the study utilized:

  • TaqMan RT-PCR dataset: Generated in the MAQC-I study, providing expression values for over 1000 genes.
  • Illumina BeadChip microarray data: Also generated in the MAQC-I study for the same samples as the RNA-seq data.

Key Insights and Ideas

  • RefSeq's Conservative Approach Wins: The study consistently found that the RefSeq annotations, particularly the older RefSeq-Rsubread, provided more accurate gene expression quantification compared to the Ensembl annotation. This suggests that a more curated and conservative approach to annotation might be beneficial for RNA-seq analysis.
  • Recent RefSeq Updates May Harm Accuracy: Surprisingly, the recent RefSeq-NCBI annotation, which incorporates newer sequencing data, did not outperform the older RefSeq-Rsubread. This raises concerns about the quality control and potential errors introduced by the rapid expansion of annotations using next-generation sequencing data.
  • Annotation Choice Affects Gene Filtering: Ensembl annotations resulted in a significantly higher proportion of genes being filtered out due to low or no expression, suggesting it might be more "aggressive" in defining genes, some of which may not be robustly expressed.
  • Effective Gene Length Differences: Ensembl annotations tend to have more genes with shorter effective lengths, but when comparing common genes, Ensembl genes often have longer effective lengths. This highlights complex differences in how gene structures are defined.
  • Subread+featureCounts Outperforms Other Tools: The study also found that the Subread aligner combined with featureCounts generally produced higher quantification accuracy than other tested methods like STAR, RSEM, and Kallisto, regardless of the annotation used.

Key Results

  • Titration Monotonicity: RefSeq-Rsubread consistently showed the lowest Mean Squared Error (MSE) when compared to the expected titration monotonicity, indicating better preservation of this ground truth.
  • TaqMan RT-PCR Correlation: RefSeq-Rsubread yielded the highest Pearson correlation coefficients between RNA-seq and RT-PCR data, outperforming both Ensembl and RefSeq-NCBI.
  • Microarray Correlation: Similarly, RefSeq-Rsubread consistently showed the highest correlation with microarray data across various comparisons and normalization methods.
  • Fragment Assignment: RefSeq-Rsubread assigned a substantially higher percentage of fragments to genes compared to Ensembl and RefSeq-NCBI, potentially due to less gene overlap and better compatibility with mapped fragments.

Limitations and Future Work

  • Concerns about Recent Annotations: The authors express concern that the rapid expansion of RefSeq annotations, driven by sequencing data, might introduce errors and reduce accuracy. They suggest that these changes need careful review and validation.
  • Impact on Single-Cell RNA-seq: The findings have implications for single-cell RNA-seq, where accurate gene annotation is also critical. The authors suggest further investigation into how annotation choice impacts single-cell data quantification.
  • Potential for Improved Multi-mapping Handling: While Subread+featureCounts' method for handling multi-mapping fragments is not ideal, its impact was minimal due to the low percentage of such fragments. However, the authors suggest that a probabilistic approach could further improve this.
  • Annotation Errors: The surprising accuracy of the older RefSeq-Rsubread annotation might be due to errors in the newer RefSeq-NCBI annotation, potentially stemming from the use of short-read sequencing data in its generation.

This study provides valuable insights for researchers using RNA-seq, emphasizing the critical role of annotation choice and highlighting the potential pitfalls of rapidly evolving annotation databases.

Summary updated

BibTeX
@article{chisanga2022impact,
  author = {David Chisanga and Yang Liao and Wei Shi},
  title = {{Impact of gene annotation choice on the quantification of RNA-seq data}},
  journal = {BMC Bioinformatics},
  year = {2022},
  doi = {10.1186/s12859-022-04644-8},
}
Figure