Guidelines for Genome-Scale Analysis of Biological Rhythms

Michael E. Hughes, Katharine C. Abruzzi, Ravi Allada, Ron C. Anafi, Alaaddin Bulak Arpat, Gad Asher, Pierre Baldi, Charissa de Bekker, Deborah Bell‐Pedersen, Justin Blau, Steven R. Brown, M. Fernanda Ceriani, Zheng Chen, Joanna C. Chiu, Jüergen Cox, T. M. Finch, Jason P. DeBruyne, Derk‐Jan Dijk, Luciano DiTacchio, Francis J. Doyle, Giles E. Duffield, Jay Dunlap, Kristin Eckel‐Mahan, Karyn A. Esser, Garret A. FitzGerald, Daniel B. Forger, Lauren J. Francey, Ying‐Hui Fu, Frédéric Gachon, David Gatfield, Paul de Goede, Susan S. Golden, Carla A. Green, John Harer, Stacey L. Harmer, Jeff Haspel, Michael H. Hastings, Hanspeter Herzel, Erik D. Herzog, Christy M. Hoffmann, Christian I. Hong, Jacob Hughey, Jennifer Hurley, Horacio O. de la Iglesia, Carl Hirschie Johnson, Steve A. Kay, Nobuya Koike, Karl Kornacker, Achim Kramer, Katja Lamia, Tanya Leise, Scott A. Lewis, Jiajia Li, Xiaodong Li, Andrew C. Liu, Jennifer Loros, Tami A. Martino, Jérôme S. Menet, Martha Merrow, Andrew J. Millar, Todd C. Mockler, Félix Naef, Emi Nagoshi, Michael N. Nitabach, María Olmedo, Dmitri A. Nusinow, Louis J. Ptáček, D.a.j. Rand, Akhilesh B. Reddy, María S. Robles, Till Roenneberg, Michael Rosbash, Marc D. Ruben, Samuel S. C. Rund, Aziz Sancar, Paolo Sassone‐Corsi, Amita Sehgal, Scott Sherrill-Mix, Debra J. Skene, Kai‐Florian Storch, Joseph S. Takahashi, Hiroki R. Ueda, Han Wang, Charles J. Weitz, Pål O. Westermark, Herman Wijnen, Ying Xu, Gang Wu, Seung Hee Yoo, Michael W. Young, Eric Erquan Zhang, T. Zieliński, John B. HogeneschJournal of Biological Rhythms 2017 · DOI

TL;DR

This paper provides a much-needed set of guidelines for analyzing biological rhythms using large-scale genomic data. It addresses the challenges of costly, complex experiments and the lack of consensus in data analysis, offering principles for reproducible and statistically sound research. The authors also introduce CircaInSilico, a tool for generating synthetic data to benchmark analytical methods.

What Problem Does This Paper Address?

The study of biological rhythms, from daily sleep-wake cycles to seasonal changes, is fundamental to understanding life. Advances in "omics" technologies (genomics, transcriptomics, proteomics, metabolomics) have revolutionized our ability to identify the molecular components that drive these rhythms. However, these genome-scale experiments generate massive datasets ("big data") that are notoriously difficult to analyze. There's a significant lack of standardized practices for experimental design and data analysis, leading to challenges in reproducibility and comparability across studies. This paper aims to bridge this gap by codifying principles for generating and analyzing high-quality, broadly useful genome-scale data on biological rhythms.

Methods

The paper doesn't introduce a single novel algorithm but rather synthesizes best practices and offers a framework for evaluating experimental designs. Key methodological considerations discussed include:

  • Experimental Design: Emphasizing the importance of defining clear research questions (discovery vs. comprehensive identification), collecting sufficient data (at least two full cycles of data under constant conditions), isolating organisms from external cues (zeitgebers) when studying intrinsic rhythms, and careful consideration of sampling frequency (recommending at least every 2 hours for circadian rhythms).
  • Data Handling: Warning against duplicating and concatenating data, which violates statistical independence and inflates false-positive rates. It also highlights the distinction between technical and biological replicates.
  • Statistical Analysis: Discussing various statistical approaches for detecting rhythmicity (e.g., Haystack, Lomb-Scargle, JTK_Cycle, RAIN) and emphasizing the critical need for controlling for multiple testing (e.g., using False Discovery Rate - FDR).
  • CircaInSilico: A novel web-based platform is introduced for generating synthetic time-series data. This tool allows users to simulate rhythmic and arrhythmic data with specified parameters (duration, sampling interval, replicates, amplitude, period length, etc.) to benchmark the performance of different analytical pipelines.

Datasets

The paper does not present results from a specific empirical dataset. Instead, it uses simulations and references existing literature to support its recommendations.

  • Synthetic Data Benchmark: The primary "dataset" introduced is the synthetic data generated by CircaInSilico. This platform serves as a benchmark for evaluating analytical methods. Users can generate data with varying parameters to test how well different algorithms perform under different experimental conditions, such as varying sampling frequency, number of replicates, and amplitude.

Key Insights and Ideas

  • Reproducibility is Paramount: The core message is that rigorous experimental design and statistical analysis are crucial for generating reproducible and reliable data in the field of biological rhythms.
  • "Big Data" Requires Careful Handling: The sheer volume of data from "-omics" studies necessitates a shift from simple binary classifications (rhythmic/arrhythmic) to a more nuanced understanding of probabilistic answers and the importance of effect sizes (amplitude, fold change).
  • Avoid Data Duplication: Concatenating time-series data to artificially lengthen it is a common pitfall that severely compromises statistical validity and leads to inflated false-positive rates.
  • Sampling Frequency Matters: For circadian rhythms, sampling at least every 2 hours is recommended to capture the dynamics accurately, especially when dealing with complex datasets.
  • Benchmarking is Essential: The introduction of CircaInSilico highlights the need for standardized tools to compare and validate analytical methods, moving beyond claims of simply detecting "more" rhythmic features.
  • Beyond Binary Classification: The paper advocates for focusing on rhythmic parameters (period, phase, amplitude) rather than solely on whether a time series is declared "rhythmic" or "arrhythmic."

Key Results

The paper's "results" are primarily in the form of recommendations and insights derived from simulations and literature review.

  • False Positive Rates: Simulations demonstrate that duplicating and concatenating time-series data can increase false-positive rates by as much as 18-fold, rendering statistical findings unreliable.
  • Sampling Density Impact: The paper argues that historical sampling densities (e.g., every 4 hours) are often statistically underpowered for modern "-omics" studies, leading to a lack of overlap in detected cycling genes.
  • CircaInSilico Capabilities: The platform can generate synthetic data with user-defined parameters, allowing for systematic testing of analytical pipelines and prediction of false-negative and false-discovery rates for various experimental designs.
  • Recommendations for Best Practices: The paper provides a comprehensive set of recommendations covering experimental design, data handling, and statistical analysis, aiming to improve the rigor and reproducibility of research in biological rhythms.

Limitations and Future Work

  • Limitations of CircaInSilico: The authors acknowledge that CircaInSilico is a starting point and does not simulate all potential complexities, such as batch effects, uneven phase distributions, trends, or "red noise," or alternatively shaped rhythms like pulses.
  • Future Work:
    • Standardizing methods for analyzing rhythmic time series from the same individual, particularly for human data.
  • Developing a standard for statistically assessing the likelihood that a data series is not rhythmic.
  • Developing and consistently applying methods for evaluating perturbations of rhythmic parameters (period, phase, amplitude).
  • The field needs more rigorous statistical definitions of arrhythmicity.
  • Further development of robust statistical methods for comparing rhythmic parameters and more general use of existing tools.
  • The paper suggests that as sequencing technologies advance, it may become cost-effective to analyze rhythmic gene expression at the individual level, requiring new statistical methods.
  • The authors encourage the development of more rigorous statistical methods for comparing rhythmic parameters and the more general use of existing tools.

Summary updated

Abstract

Genome biology approaches have made enormous contributions to our understanding of biological rhythms, particularly in identifying outputs of the clock, including RNAs, proteins, and metabolites, whose abundance oscillates throughout the day. These methods hold significant promise for future discovery, particularly when combined with computational modeling. However, genome-scale experiments are costly and laborious, yielding "big data" that are conceptually and statistically difficult to analyze. There is no obvious consensus regarding design or analysis. Here we discuss the relevant technical considerations to generate reproducible, statistically sound, and broadly useful genome-scale data. Rather than suggest a set of rigid rules, we aim to codify principles by which investigators, reviewers, and readers of the primary literature can evaluate the suitability of different experimental designs for measuring different aspects of biological rhythms. We introduce CircaInSilico, a web-based application for generating synthetic genome biology data to benchmark statistical methods for studying biological rhythms. Finally, we discuss several unmet analytical needs, including applications to clinical medicine, and suggest productive avenues to address them.
BibTeX
@article{hughes2017guidelines,
  author = {Michael E. Hughes and Katharine C. Abruzzi and Ravi Allada and Ron C. Anafi and Alaaddin Bulak Arpat and Gad Asher and Pierre Baldi and Charissa de Bekker and Deborah Bell‐Pedersen and Justin Blau and Steven R. Brown and M. Fernanda Ceriani and Zheng Chen and Joanna C. Chiu and Jüergen Cox and T. M. Finch and Jason P. DeBruyne and Derk‐Jan Dijk and Luciano DiTacchio and Francis J. Doyle and Giles E. Duffield and Jay Dunlap and Kristin Eckel‐Mahan and Karyn A. Esser and Garret A. FitzGerald and Daniel B. Forger and Lauren J. Francey and Ying‐Hui Fu and Frédéric Gachon and David Gatfield and Paul de Goede and Susan S. Golden and Carla A. Green and John Harer and Stacey L. Harmer and Jeff Haspel and Michael H. Hastings and Hanspeter Herzel and Erik D. Herzog and Christy M. Hoffmann and Christian I. Hong and Jacob Hughey and Jennifer Hurley and Horacio O. de la Iglesia and Carl Hirschie Johnson and Steve A. Kay and Nobuya Koike and Karl Kornacker and Achim Kramer and Katja Lamia and Tanya Leise and Scott A. Lewis and Jiajia Li and Xiaodong Li and Andrew C. Liu and Jennifer Loros and Tami A. Martino and Jérôme S. Menet and Martha Merrow and Andrew J. Millar and Todd C. Mockler and Félix Naef and Emi Nagoshi and Michael N. Nitabach and María Olmedo and Dmitri A. Nusinow and Louis J. Ptáček and D.A.J. Rand and Akhilesh B. Reddy and María S. Robles and Till Roenneberg and Michael Rosbash and Marc D. Ruben and Samuel S. C. Rund and Aziz Sancar and Paolo Sassone‐Corsi and Amita Sehgal and Scott Sherrill-Mix and Debra J. Skene and Kai‐Florian Storch and Joseph S. Takahashi and Hiroki R. Ueda and Han Wang and Charles J. Weitz and Pål O. Westermark and Herman Wijnen and Ying Xu and Gang Wu and Seung Hee Yoo and Michael W. Young and Eric Erquan Zhang and T. Zieliński and John B. Hogenesch},
  title = {{Guidelines for Genome-Scale Analysis of Biological Rhythms}},
  journal = {Journal of Biological Rhythms},
  year = {2017},
  doi = {10.1177/0748730417728663},
}
Figure