Abstract
In 2013, a comparative analysis of mutation and gene expression profiles and drug sensitivity measurements for 15 drugs across 471 cancer cell lines revealed substantial inconsistency in the drug responses reported by the Genomics of Drug Sensitivity in Cancer (GDSC) and Cancer Cell Line Encyclopedia (CCLE) projects. This re-analysis addresses feedback and incorporates expanded data, focusing on the distinction between targeted therapies and broad cytotoxic drugs, and the consistency of molecular profiles versus pharmacological phenotypes. Our findings confirm that while gene expression data are significantly more consistent, drug sensitivity measurements continue to pose challenges for robust biomarker discovery. The use of new statistics allowed the identification of five drugs with moderate to good consistency, but inconsistencies remain for the majority. Overall, this work emphasizes the need for experimental standardization and validation to advance the use of large pharmacogenomic screens.
Introduction: The Reproducibility Challenge in Pharmacogenomics
The promise of precision oncology relies on identifying molecular biomarkers that predict drug sensitivity. Large-scale screens like the GDSC and CCLE are foundational to this effort. However, initial comparisons suggested significant disagreements between these datasets regarding which cell lines respond to specific compounds. This study revisits those findings using updated data and refined statistical methodologies to determine if these inconsistencies are inherent to the biological assays or a result of methodological differences.
Methods and Data Integration
To ensure a rigorous comparison, the researchers developed PharmacoGx, an open-source R package designed to standardize the processing of large-scale pharmacogenomic data. The analysis focused on 15 drugs and 698 cell lines common to both the July 2015 GDSC release and the February 2015 CCLE release.
Key methodological improvements included:
- Standardized Curve Fitting: Raw dose-response data were re-fitted using a consistent three-parameter sigmoid model to eliminate discrepancies introduced by different original mathematical models.
- Area Between Curves (ABC): This metric was employed to quantify geometric differences between response curves over identical concentration ranges.
- SNP Fingerprinting: Genetic identity was verified across studies to rule out cell line drift or mislabeling.
Key Insights and Results
The re-analysis highlighted a stark contrast between molecular and phenotypic robustness. While gene expression and copy number data showed high consistency (Spearman correlation > 0.8), drug sensitivity measurements remained highly sensitive to experimental noise.
Targeted vs. Broad-Effect Drugs
Consistency was notably higher for highly targeted therapies, such as PLX4720 (targeting BRAF mutations), where the "all-or-nothing" biological response provides a signal strong enough to overcome experimental variability. In contrast, broad cytotoxic drugs exhibited significant noise, likely due to differences in experimental conditions such as seeding density or growth media.
Biomarker Validation
Out of the 15 drugs studied, only five (Nilotinib, Crizotinib, PLX4720, 17-AAG, and PD-0325901) showed moderate to good consistency. However, known biomarkers were successfully validated in both datasets. By applying more stringent statistics, the authors achieved a >75% validation rate for the most significant biomarkers of eight drugs, a marked improvement over previous analyses.
Discussion and Future Directions
Despite improvements in validation rates, the study underscores that many discovered associations remain dataset-specific. Limitations include the lack of raw technical triplicate data from CCLE and the inherent arbitrariness of sensitivity thresholds (e.g., IC50 cutoffs).
The authors conclude that the path forward for precision medicine requires the community to adopt standardized experimental and computational pipelines. Until laboratory protocols for drug screening are harmonized, the potential for large-scale pharmacogenomics to drive clinical breakthroughs will remain constrained by the challenges of reproducibility.
Summary updated