Benchmark of lncRNA Quantification for RNA-Seq of Cancer Samples

Hong Zheng, Kevin Brennan, Mikel Hernáez, Olivier GevaertbioRxiv 2018 · DOI

TL;DR

This paper benchmarks different RNA-Seq quantification methods for long non-coding RNAs (lncRNAs) in cancer. It finds that pseudoalignment methods (Kallisto and Salmon) combined with full transcriptome annotations are superior to traditional alignment-based methods, especially for detecting and accurately quantifying lncRNAs.

What Problem Does This Paper Address?

Long non-coding RNAs (lncRNAs) are increasingly recognized as crucial regulators in biological processes, including cancer. Researchers often use RNA-Seq data, particularly from large public repositories like The Cancer Genome Atlas (TCGA), to study lncRNA expression. However, accurately quantifying lncRNA expression from RNA-Seq data is challenging due to their unique characteristics (e.g., lower expression levels, shorter transcripts, and less evolutionary conservation compared to protein-coding genes). Different computational tools exist for RNA-Seq analysis, but there's no clear consensus on which ones perform best for lncRNAs. This lack of a gold standard can lead to inconsistent and potentially inaccurate findings, hindering our understanding of lncRNA roles in cancer.

Methods

The study compares five popular RNA-Seq quantification methods:

  • Pseudoalignment methods: Kallisto and Salmon. These methods assign reads to a set of compatible transcripts without performing a full alignment to the genome, making them faster.
  • Alignment-based methods: HTSeq, featureCounts, and RSEM. These methods first align reads to the genome or transcriptome and then count them.

The researchers applied these methods to both simulated RNA-Seq data and a real pan-cancer RNA-Seq dataset from TCGA. They evaluated performance based on:

  • Specificity: How accurately the methods identify expressed lncRNAs.
  • Concordance with ground truth: How well the quantified expression levels match simulated "true" values.
  • Detection rate: The percentage of lncRNAs identified as expressed.
  • Discordance: The extent to which a method's results differ from the ground truth.

Datasets

The study utilized two main types of datasets:

  1. Simulated RNA-Seq dataset: Generated using RSEM based on 63 TCGA samples from 21 cancer types, with varying levels of simulated noise (5%, 15%, and 30%) to mimic real-world data variability.
  2. Pan-cancer RNA-Seq dataset: A real dataset comprising 210 TCGA samples from 21 cancer types.

The transcriptome annotation used was from GENCODE release 25, which includes both protein-coding genes and lncRNAs.

Key Insights and Ideas

  • Full transcriptome annotation is crucial: Using an annotation that includes all transcripts (protein-coding and non-coding) significantly improves the accuracy and specificity of lncRNA quantification, preventing reads from being falsely assigned to lncRNAs.
  • Pseudoalignment methods shine for lncRNAs: Kallisto and Salmon consistently outperform alignment-based methods (HTSeq, featureCounts, RSEM) in detecting more lncRNAs and correlating better with the simulated ground truth.
  • Alignment-based methods underestimate lncRNAs: These methods tend to underestimate lncRNA expression or even miss them entirely, including important cancer-related lncRNAs like TERC and ZEB2-AS1.
  • Antisense lncRNAs are particularly tricky: Antisense lncRNAs are more prone to misquantification by alignment-based methods, contributing significantly to discordant results.
  • lncRNA features matter: lncRNAs with fewer transcripts, fewer exons, and lower sequence uniqueness are more likely to be discordant, especially when analyzed with alignment-based methods.

Key Results

  • Detection Rates: Pseudoalignment methods (Kallisto, Salmon) detected approximately 16% of annotated lncRNAs in the pan-cancer dataset, while alignment-based methods detected fewer (10-13%).
  • Concordance: Pseudoalignment methods showed much higher concordance with ground truth compared to alignment-based methods. For example, among expressed lncRNAs, only 2 were discordant for Kallisto/Salmon, versus 187 and 143 for HTSeq and featureCounts, respectively.
  • Speed: While not explicitly quantified in detail for all comparisons, the paper notes that pseudoalignment methods are generally faster due to their alignment-free nature.
  • Cancer-Relevant lncRNAs: The study highlighted that alignment-based methods often failed to capture or underestimated the expression of known cancer-relevant lncRNAs like TERC and HOTAIR.
  • lncRNA Categories: Antisense lncRNAs and lincRNAs were the most abundant detected categories, with antisense lncRNAs being more prevalent in the expressed set than in the overall annotation.

Limitations and Future Work

  • TCGA Default Workflow: The paper points out that HTSeq, an alignment-based method, is the default workflow for TCGA data in the GDC portal. This suggests that re-analyzing existing TCGA data using recommended methods might be beneficial.
  • Transcript-level vs. Gene-level: The study primarily focused on gene-level analysis. For transcript-level expression analysis, HTSeq and featureCounts are not suitable, and Kallisto, Salmon, or RSEM should be used.
  • Suboptimal performance in specific cases: While pseudoalignment methods generally perform best, they can still be suboptimal for certain cancer-relevant lncRNAs in specific samples. The paper suggests that even with these methods, accurate quantification isn't always guaranteed for all lncRNAs in all samples.
  • Future work: The authors recommend using pseudoalignment methods (Kallisto or Salmon) in conjunction with full transcriptome annotation as the preferred strategy for lncRNA analysis. They also highlight the importance of further investigating the abundant antisense RNAs in cancer samples.

Summary updated

Abstract

ABSTRACT Long non-coding RNAs (lncRNAs) emerge as important regulators of various biological processes. Many lncRNAs with tumor-suppressor or oncogenic functions in cancer have been discovered. While many studies have exploited public resources such as RNA-Seq data in The Cancer Genome Atlas (TCGA) to study lncRNAs in cancer, it is crucial to choose the optimal method for accurate expression quantification of lncRNAs. In this benchmarking study, we compared the performance of pseudoalignment methods Kallisto and Salmon, and alignment-based methods HTSeq, featureCounts, and RSEM, in lncRNA quantification, by applying them to a simulated RNA-Seq dataset and a pan-cancer RNA-Seq dataset from TCGA. We observed that full transcriptome annotation, including both protein coding and noncoding RNAs, greatly improves the specificity of lncRNA expression quantification. Pseudoalignment-based methods detect more lncRNAs than alignment-based methods and correlate highly with simulated ground truth. On the contrary, alignment-based methods tend to underestimate lncRNA expression or even fail to capture lncRNA signal in the ground truth. These underestimated genes include cancer-relevant lncRNAs such as TERC and ZEB2-AS1 . Overall, 10–16% of lncRNAs can be detected in the samples, with antisense and lincRNAs the two most abundant categories. A higher proportion of antisense RNAs are detected than lincRNAs. Moreover, among the expressed lncRNAs, more antisense RNAs are discordant from ground truth than lincRNAs when measured by alignment-based methods, indicating that antisense RNAs are more susceptible to mis-quantification. In addition, the lncRNAs with fewer transcripts, less than three exons, and lower sequence uniqueness tend to be more discordant. In summary, pseudoalignment methods Kallisto or Salmon in combination with the full transcriptome annotation is our recommended strategy for RNA-Seq analysis for lncRNAs. AUTHOR SUMMARY Long non-coding RNAs (lncRNAs) emerge as important regulators of various biological processes. Our benchmarking work on both simulated RNA-Seq dataset and pan-cancer dataset provides timely and useful recommendations for wide research community who are studying lncRNAs, especially for those who are exploring public resources such as TCGA RNA-Seq data. We demonstrate that using full transcriptome annotation in RNA-Seq analysis is strongly recommended as it greatly improves the specificity of lncRNA quantification. What’s more, pseudoalignment methods Kallisto and Salmon outperform alignment-based methods in lncRNA quantification. It is worth noting that the default workflow for TCGA RNA-Seq data stored in Genomic Data Commons (GDC) data portal uses HTSeq, an alignment-based method. Thus, reanalyzing the data might be considered when checking gene expression in TCGA datasets. In summary, pseudoalignment methods Kallisto or Salmon in combination with full transcriptome annotation is our recommended strategy for RNA-Seq analysis for lncRNAs.
BibTeX
@misc{zheng2018benchmark,
  author = {Hong Zheng and Kevin Brennan and Mikel Hernáez and Olivier Gevaert},
  title = {{Benchmark of lncRNA Quantification for RNA-Seq of Cancer Samples}},
  year = {2018},
  doi = {10.1101/241869},
  howpublished = {bioRxiv},
}

Loading related content...

Figure