Abstract
Background: Previous measurement error work that investigates the relationship between a nutritional biomarker and self-reported intake levels has typically been at a single time point, in a single treatment group, or with respect to basic patient demographics. Few studies have examined the measurement error structure in longitudinal randomized trials, and whether the error varies across time or group. This structure is crucial to understand, however, in order to correct for measurement error in self-reported outcomes and properly interpret the longitudinal effects of dietary interventions. Methods: Using two longitudinal randomized controlled trials with internal longitudinal validation data (urinary biomarkers and self-reported values), we examine the relationship between urinary sodium and self-reported sodium and whether this relationship changes as a function of time and/or treatment condition. We do this by building a mixed effects regression model, allowing for a flexible error variance-covariance structure, and testing all possible interactions between time, treatment condition, and self-reported intake. Results: Using a backward selection approach, we arrived at the same final model for both validation data sets. We found no evidence that measurement error changes as a function of self-reported sodium. However, we did find evidence that urinary sodium can differ by time or treatment condition even when conditioning on self-reported values. Conclusion: In longitudinal nutritional intervention trials it is possible that measurement error differs across time and treatment groups. It is important for researchers to consider this possibility and not just assume non-differential measurement error. Future studies should consider data collection strategies to account for the potential dynamic nature of measurement error, such as collecting internal validation data across time and treatment groups when possible.
Introduction
Nutritional research faces a significant challenge regarding data reliability. Human memory and social desirability bias often make self-reported dietary data, such as 24-hour recalls, notoriously inaccurate. To mitigate this, researchers frequently use objective biomarkers—like 24-hour urinary sodium—to calibrate self-reports.
However, most calibration models rely on the assumption of non-differential measurement error. This assumes that the relationship between reported intake and actual intake remains constant throughout a study, regardless of time or whether a participant is in a control or intervention group. If an intervention makes a participant more mindful of their salt intake, they might become more accurate, or conversely, they might under-report to appear more compliant. If this error structure shifts during the study, it can lead to biased estimates of the intervention's true effectiveness.
Methods and Data Sources
This study employed a calibration model approach using mixed-effects linear regression to regress a reference measure (24-hour urinary sodium) on self-reported 24-hour dietary recalls. Both measures were natural log-transformed to normalize distributions. The model included random intercepts and slopes to account for individual-level variation in reporting habits over time.
Researchers used a backward variable selection process, starting with a saturated model including three-way interactions between time, treatment status, and self-reported intake. This allowed them to identify the most parsimonious model explaining the error structure across two landmark datasets:
- TOHP (Trials of Hypertension Prevention, Phase I): Focused on sodium reduction and weight loss (n=751).
- PREMIER: Tested lifestyle interventions for blood pressure control, including the DASH diet (n=818).
Both trials are unique for collecting both self-reports and urinary biomarkers at multiple time points (baseline, 6 months, and 18 months).
Key Results and Insights
Differential Error in the TOHP Study
In the TOHP study, the final model revealed that measurement error was indeed differential. While the slope of the relationship between reported and actual sodium remained stable, the intercept shifted significantly for the treatment group.
At the 6-month mark, for any given level of self-reported sodium, the treatment group (those receiving sodium reduction counseling) had 0.81 SD lower actual urinary sodium than the control group ($\beta_4 = -0.81, p < 0.001$). This gap persisted at 18 months ($\beta_5 = -0.65, p < 0.001$), indicating that the intervention group's reporting habits changed relative to the biomarker.
Dynamic Error in the PREMIER Study
The PREMIER study also showed evidence of differential error, though the patterns differed from TOHP. In PREMIER, there was a significant decrease in actual urinary sodium relative to self-reports at 6 months across both groups ($\beta_2 = -0.24, p < 0.001$), but this difference largely faded by 18 months.
These findings suggest that measurement error is dynamic and study-specific. In TOHP, the intensive counseling likely drove the gap between treatment and control reporting, whereas in PREMIER, the effect was more generalized across the study population over time.
Discussion and Conclusion
The primary takeaway is that the "honesty" of self-reporting is not a fixed variable. If researchers had ignored these shifts and assumed non-differential error, they would have likely miscalculated the true impact of the sodium reduction interventions.
Limitations
- Missing Data: Significant attrition (up to 29% at 18 months in TOHP) required an assumption of "Missing at Random," which may not hold if dropouts had different reporting habits.
- Timing Mismatch: Dietary recalls and urine collections were not always performed on the same day, introducing biological noise.
- Biomarker Accuracy: 24-hour urine captures approximately 86% of sodium intake and has its own inherent measurement error.
Future Directions
To properly interpret longitudinal effects, future nutritional intervention trials must move away from assuming static error structures. It is essential to collect internal validation data (biomarkers) at multiple time points across both treatment and control groups to accurately calibrate self-reported outcomes.
Summary updated