Repeated Measures Correlation

Jonathan Z. Bakdash, Laura R. Marusich

TL;DR

This paper introduces rmcorr, a statistical technique designed to accurately measure the linear association between two variables when data comes from repeated observations of the same individuals. Unlike traditional methods that can produce biased results due to non-independent data, rmcorr respects this dependency, offering greater statistical power and more accurate insights into within-individual relationships.

What Problem Does This Paper Address?

Many research questions involve tracking the same individuals over time or across multiple conditions, leading to "repeated measures" data. Standard correlation and regression techniques, like Pearson correlation, assume that each data point is independent. When applied to repeated measures data, this assumption is violated, potentially leading to misleading or "specious" results. A common workaround is to average the data for each individual, but this can obscure important individual differences and reduce statistical power. The authors highlight the need for a method that can directly analyze paired repeated measures data while accounting for the inherent dependency, allowing researchers to investigate intra-individual associations without sacrificing power or introducing bias.

Methods

The core of the paper is the introduction and explanation of the Repeated Measures Correlation (rmcorr) technique. rmcorr is presented as a specialized application of Analysis of Covariance (ANCOVA). Instead of using ANCOVA to control for a "nuisance" variable to assess group differences, rmcorr uses ANCOVA to assess the common linear association between two continuous variables while statistically controlling for between-participant variance. This is achieved by fitting parallel regression lines (with the same slope) to each individual's data, effectively estimating the common intra-individual association. The paper details the assumptions of rmcorr (linearity, normality of errors, but crucially, not independence of errors), its equations, and provides visualizations (rmcorr plots) to illustrate its application. They also introduce an R package, rmcorr, to make the technique accessible.

Datasets

The paper uses two publicly available datasets to demonstrate the rmcorr package and its application:

  1. Raz et al. (2005) dataset: This dataset contains repeated measures of age and cerebellar hemisphere brain (CBH) structural volume for individuals assessed approximately 5 years apart.
  2. Gilden et al. (2010) dataset: This dataset includes average reaction time (RT) and accuracy for repeated blocks of visual search trials from 11 participants.

These datasets were chosen to illustrate intra-individual research questions and to compare rmcorr results with those from traditional methods.

Key Insights and Ideas

  • Respecting Dependency is Crucial: Violating the independence assumption in repeated measures data can lead to seriously misleading conclusions. rmcorr directly addresses this by modeling the within-individual association.
  • Power Boost: By utilizing all available data points per individual without averaging, rmcorr generally offers significantly higher statistical power compared to methods that aggregate data. This means it's more likely to detect real associations.
  • Intra- vs. Inter-Individual Associations: The paper strongly emphasizes the distinction between within-individual (intra-individual) and between-individual (inter-individual) associations. rmcorr is specifically designed for the former, while standard regression often addresses the latter. These can be very different, as illustrated by the examples.
  • Visualization is Key: The rmcorr plot, which shows individual regression lines, is a powerful tool for understanding the nature of the within-individual association and identifying potential issues like non-linear relationships or heterogeneity.
  • rmcorr vs. Multilevel Modeling: While rmcorr is a powerful tool for paired repeated measures, it's presented as a "lighter" version of multilevel modeling. For more complex designs with varying slopes or multiple levels of analysis, multilevel modeling offers greater flexibility but requires more data and expertise.

Key Results

  • Age and Brain Volume: In the Raz et al. dataset, rmcorr revealed a stronger negative association between age and CBH volume (rrm = -0.70) at the intra-individual level compared to cross-sectional analyses (r = -0.36 to -0.40). This highlights how longitudinal changes can differ from cross-sectional patterns.
  • Speed-Accuracy Tradeoff: In the visual search task, rmcorr identified a significant negative intra-individual association between reaction time and accuracy (rrm = -0.41), consistent with a speed-accuracy tradeoff. Averaging data or treating observations as independent yielded different, and in one case, uninterpretable, results.
  • Power Advantage: The paper includes power curves demonstrating that rmcorr generally achieves higher power than Pearson correlation with averaged data, especially as the number of repeated measures (k) or participants (N) increases.

Limitations and Future Work

  • Linearity Assumption: rmcorr assumes a linear relationship between variables. If the relationship is non-linear, transformations or more complex models (like nonlinear multilevel models) might be necessary.
  • Reliability: The accuracy of rmcorr can be affected by the reliability of the measures. The authors suggest that corrections for attenuation could be applied if reliability is known.
  • Complexity vs. Simplicity: While rmcorr is simpler than multilevel modeling, for very complex datasets or research questions involving varying slopes, multilevel modeling remains the more flexible and powerful option.
  • Future Work: The authors plan to expand the rmcorr package with more examples and functionality. They suggest rmcorr could complement multilevel modeling, potentially by assessing collinearity in null multilevel models or providing effect sizes for such models. Further research is also planned to investigate the stability of the rmcorr coefficient across different sample sizes and effect sizes.

Summary updated

Abstract

Repeated measures correlation (rmcorr) is a statistical technique for determining the common within-individual association for paired measures assessed on two or more occasions for multiple individuals. Simple regression/correlation is often applied to non-independent observations or aggregated data; this may produce biased, specious results due to violation of independence and/or differing patterns between-participants versus within-participants. Unlike simple regression/correlation, rmcorr does not violate the assumption of independence of observations. Also, rmcorr tends to have much greater statistical power because neither averaging nor aggregation is necessary for an intra-individual research question. Rmcorr estimates the common regression slope, the association shared among individuals. To make rmcorr accessible, we provide background information for its assumptions and equations, visualization, power, and tradeoffs with rmcorr compared to multilevel modeling. We introduce the R package (rmcorr) and demonstrate its use for inferential statistics and visualization with two example datasets. The examples are used to illustrate research questions at different levels of analysis, intra-individual, and inter-individual. Rmcorr is well-suited for research questions regarding the common linear association in paired repeated measures data. All results are fully reproducible.
BibTeX
@inproceedings{bakdash2017repeated,
  author = {Jonathan Z. Bakdash and Laura R. Marusich},
  title = {{Repeated Measures Correlation}},
  booktitle = {Frontiers in Psychology},
  year = {2017},
  doi = {10.3389/fpsyg.2017.00456},
}
Figure