Abstract: Micro-randomized trials (MRTs), which sequentially randomize participants at multiple decision times, have gained prominence in digital intervention development. These sequential randomizations are often subject to certain constraints. In the MRT called HeartSteps V2V3, where an intervention is designed to interrupt sedentary behavior, two core design constraints need to be managed: an average of 1.5 interventions across days and the uniform delivery of interventions across decision times. Meeting both constraints, especially when the times allowed for randomization are not determined beforehand, is challenging. An online algorithm was implemented to meet these constraints in the HeartSteps V2V3 MRT. We present a case study using data from the HeartSteps V2V3 MRT, where we select appropriate metrics, discuss issues in making an accurate evaluation, and assess the algorithm's performance. Our evaluation shows that the algorithm performed well in meeting the two constraints. Furthermore, we identify areas for improvement and provide recommendations for designers of MRTs that need to satisfy these core design constraints.
Introduction
In the field of digital health, "nudging" a user to engage in healthy behaviors—such as standing up or taking a walk—requires a delicate balancing act. Delivering too many notifications can lead to "notification fatigue," while delivering them at the wrong times can result in missed opportunities to learn when interventions are most effective. This paper evaluates SeqRTS (Sequential Risk Time Sampling), an online algorithm designed to manage this balance in real-time during the HeartSteps V2V3 micro-randomized trial (MRT).
The Challenge of Real-Time Randomization
Micro-randomized trials are the gold standard for developing "just-in-time" adaptive interventions, randomizing treatments thousands of times over a study period. However, researchers face two major design constraints:
- Average Treatment Constraint: To avoid user burden, researchers often set a target for the average number of messages sent per day (e.g., 1.5 messages).
- Uniformity Constraint: To ensure unbiased data collection across all contexts (e.g., morning vs. evening, stressed vs. calm), messages should be distributed evenly across all "risk times"—moments when a user is sedentary and available for an intervention.
Managing these constraints is difficult because the total number of risk times in a day is unknown until the day is over. If an algorithm uses its "budget" too early, it cannot sample in the evening; if it waits too long, it may end the day under-treated.
Methodology
The authors evaluate the Sequential Risk Time Sampling (SeqRTS) algorithm, which makes randomization decisions on the fly. At each decision point, the algorithm estimates the number of remaining risk times in the current four-hour block ($\hat{g}_t$). It then calculates a randomization probability by dividing the remaining message budget by this estimate.
Evaluation Metrics
A key contribution of this paper is the proposal of Mean Absolute Deviation (MAD) as a metric for uniformity. While previous studies used Kullback-Leibler (KL) divergence, the authors argue that MAD provides a cleaner measure of how much randomization probabilities fluctuate, whereas KL divergence can be confounded by whether the algorithm hit its total treatment target.
Dataset: HeartSteps V2V3
The study utilizes data from the HeartSteps V2V3 (HSV2V3) trial, involving 82 adults newly diagnosed with stage 1 hypertension. The dataset includes:
- 9,423 eligible participant days.
- 1.3 million decision points (sampled every 5 minutes).
- 166,742 identified risk times (moments where the participant was sedentary for at least 40 minutes and available for notification).
- 10,137 anti-sedentary messages delivered.
Key Results and Insights
Target Accuracy and Uniformity
The algorithm performed remarkably well at the population level. While the goal was 1.5 treatments per day, the actual average achieved was 1.486 (95% CI: 1.365, 1.607). Furthermore, MAD values remained low (ranging from 0.01 to 0.018), indicating that the message budget was spread fairly evenly across the day despite the unpredictable nature of human behavior.
The "Oracle" Comparison
In simulations, SeqRTS actually outperformed an "Oracle" version of the algorithm—one with perfect knowledge of the future. This is because real-world constraints, such as a mandatory one-hour "lockout" period after any message is sent, reduce future sampling opportunities. SeqRTS’s tuning parameters allowed it to slightly "over-budget" to compensate for these lost opportunities.
Participant Heterogeneity
Using Classification and Regression Trees (CART), the authors found that the algorithm's performance varied by participant type. The algorithm consistently over-treated highly sedentary individuals (~2 messages/day) and under-treated highly active individuals (~1.1 messages/day). This suggests that a "one-size-fits-all" approach to algorithm tuning may be insufficient.
Discussion and Recommendations
Limitations
A primary challenge in evaluation is the "responsiveness trap." If a user stands up immediately because of a message, it changes their future availability, entangling user behavior with algorithm performance. Future research must find ways to disentangle these factors.
Future Directions
Based on the evaluation, the authors provide several recommendations for future MRT designers:
- Hyperparameter Stratification: Algorithms should be "warm-started" by categorizing participants based on baseline activity levels (e.g., "High Activity" vs. "High Sedentary") and applying tailored tuning parameters to each group.
- Improved Data Logging: Researchers should maintain distinct indicators for why a user was unavailable (e.g., technical glitch vs. user activity) to facilitate more accurate post-hoc evaluations.
- Robustness: Despite a timezone conversion bug affecting 5.4% of the data, the core logic of SeqRTS remained robust enough to meet primary design goals, highlighting the importance of building resilient online algorithms.
Summary updated