Abstract
Time plays a fundamental role in causal analyses, where the goal is to quantify the effect of a specific treatment on future outcomes. In a randomized experiment, times of treatment, and when outcomes are observed, are typically well defined. In an observational study, treatment time marks the point from which pre-treatment variables must be regarded as outcomes, and it is often straightforward to establish. Motivated by a natural experiment in online marketing, we consider a situation where useful conceptualizations of the experiment behind an observational study of interest lead to uncertainty in the determination of times at which individual treatments take place. Of interest is the causal effect of heavy snowfall in several parts of the country on daily measures of online searches for batteries, and then purchases. The data available give information on actual snowfall, whereas the natural treatment is the anticipation of heavy snowfall, which is not observed. In this article, we introduce formal assumptions and inference methodology centered around a novel notion of plausible time of treatment. These methods allow us to explicitly bound the last plausible time of treatment in observational studies with unknown times of treatment, and ultimately yield valid causal estimates in such situations.
The Challenge of Fuzzy Timelines in Causal Inference
In causal inference, timing is paramount. To establish that "A caused B," researchers typically need to pinpoint exactly when "A" occurred to distinguish between pre-treatment and post-treatment states. However, in many observational studies, the treatment is not a discrete, recorded event but rather something latent, such as a person’s anticipation of a future event.
Standard models, like the Rubin Causal Model, assume a clear "time zero." In a clinical trial, this is the moment a drug is administered. In "natural experiments," we often only observe a proxy event (e.g., a snowstorm) rather than the true treatment (e.g., the expectation of snow). As shown in the conceptual timeline below, the actual treatment effect may begin well before the event is formally recorded or observed.
If a researcher only analyzes data following the proxy event, they may miss the most significant causal effects. Conversely, looking too far back risks introducing noise from unrelated factors. The challenge lies in identifying the precise moment the "treated" and "control" groups began to diverge.
Methodology: Identifying the Causal Window
To address this uncertainty, the authors propose a framework centered on the Last Plausible Randomized Experiment Time (LaPRET). This is defined as the latest time point at which the treated and control units remain statistically indistinguishable.
The methodology follows a three-stage pipeline:
- Conceptualization: Define the treatment as a perceived event and identify a correlated proxy event.
- The Pilot Study: To prevent data snooping, a subset of the data is used to estimate the lag parameter $d$—the number of days before the proxy event that the behavioral shift began. This identifies the $T^{LaPRET}$.
- The Main Study: This estimated lag $d$ is then applied to the remaining data. Data before $T^{LaPRET}$ serves as covariates, while data after is treated as the outcome.
The core technique utilizes a maximization formula that identifies the point where the difference between groups is minimal, but the rate of change in that difference is poised to increase significantly.
Datasets and Evaluation
The framework was evaluated using both synthetic simulations and real-world online marketing data:
- Synthetic Simulations: Three scenarios were tested: a non-negative effect that vanishes at the event time, a sustained positive effect, and a "volatile" curve where the effect dips (e.g., consumers stop searching because they have already purchased).
- Online Marketing Data: Using data from MaxPoint Interactive Inc. (Feb–March 2015), the authors analyzed battery searches across 79 US Designated Marketing Areas (DMAs). To account for weather heterogeneity, DMAs were subdivided into 3,676 synthetic "tradezones" paired with daily snowfall records and census demographics.
Key Insights and Results
A critical finding is that the method is conservative by design. As noise in the data increases, the estimated causal window ($d$) shrinks. This prevents the model from making unsubstantiated causal claims when the signal is weak.
In the snowfall study, the model identified a causal window of up to two days prior to the actual event. The data revealed a clear progression:
- Three days before the snow, the change in searches was near zero.
- Two days before, searches jumped by 0.95.
- One day before, the increase peaked at 2.55.
- Upon the arrival of the snow, search rates dropped sharply (-7.8), confirming that the behavioral shift (anticipation and preparation) had already concluded.
Limitations and Future Directions
While this research provides a robust "pre-processing" step for natural experiments, it has limitations. It assumes a relatively constant lag across different regions or populations, whereas real-world anticipation may vary. Additionally, splitting data for a pilot study reduces overall statistical power. Finally, as an observational method, it remains subject to potential unobserved confounders.
Nonetheless, this framework offers a formal, statistically sound method for determining exactly when the "causal clock" starts ticking in complex, real-world scenarios.
Summary updated