Measuring Regulatory Lag as a Response to Outside Shocks

Lawmakers are often said to be "behind" artificial intelligence, but that lag is almost never measured as a number with a timescale attached. This proposal treats the progress of AI bills through U.S. state legislatures as an event stream. Sudden outside developments, such as a widely publicized AI incident, a major model release, or a neighboring state passing a law, can speed that stream up. We then ask two questions. First, do these time-stamped outside signals predict when a bill is introduced, clears committee, or is enacted, beyond what fixed state characteristics already predict? Second, if they do, how many weeks pass between a shock and the legislative response? The size and shape of that delay is a working definition of regulatory lag that can be estimated and checked against data.

The forecasting literature has a name for these outside signals: exogenous variables, meaning inputs that come from outside the system being predicted, such as weather or holidays in a traffic model. That literature shows that models which only extrapolate a series' own history miss sudden disruptions. Governance research, for its part, has so far related bill outcomes only to static state attributes. We combine the two in a deliberately small model, because the data are small: 1,419 bills, and only 18 catastrophic-safety bills. We then test whether each added ingredient earns its place under a strict forward-in-time evaluation.

Core Claim

Our central claim is that regulatory lag can be written as a response function. Legislative activity in a state rises some weeks after an outside shock, and both the height and the timing of that rise can be estimated. Estimating it requires three things the current empirical work lacks: outside signals that vary over time rather than annual snapshots, a way for one state's action to influence its neighbors, and evaluation that predicts the future rather than explaining the past. The deliverable is a set of estimated delays with uncertainty, broken out by legislative stage and bill topic. A companion forecasting result would show that these delays improve out-of-sample prediction of bill progress.

Concretely, time is divided into weeks $t$. We model the probability $h_{b}(t)$ that bill $b$ in state $s$ advances to its next stage (committee to floor, or floor to enactment) in week $t$, given that it has not advanced yet:

$$ \text{logit}\, h_{b}(t) = \alpha_{k}(t) + \beta^\top x_{s} + \sum_{j} \sum_{\ell=0}^{L} w_{j,\ell}\, z_{j}(t-\ell) + \gamma \sum_{s'} A_{s s'}\, e_{s'}(t-1). $$

The terms are as follows. $\alpha_k(t)$ is a baseline for stage $k$ that also captures the legislative calendar. $x_s$ holds the static state covariates used by Khan et al. $z_j(t)$ is the weekly count of outside signal type $j$, such as incidents or major model releases. $w_{j,\ell}$ is the response kernel over lags up to $L$ weeks. $A_{ss'}$ is a between-state influence weight. $e_{s'}(t-1)$ records whether state $s'$ enacted a related bill in the previous week. The kernel $w_{j,\cdot}$ is the object of interest: its peak location and its total mass are our lag estimates. A matching count model handles bill introductions.

Relation to Prior Work

The closest work is The Bottleneck in AI Governance and Oversight: Evidence from 1,419 State Bills. It fits a two-stage hurdle model, with logistic regressions for reaching the floor and then for enactment. Its covariates are the Squire index of legislative professionalism, Stanford HAI state AI economic indicators, and partisan control. That paper supplies our data and our strongest baseline. It treats each bill as a static cross-sectional record and each state as independent, so it cannot say anything about timing or diffusion between states. Its finding that bills stall in committee tells us which stage transition the lag analysis should focus on.

The method comes from forecasting work on exogenous inputs. Zheng et al. route weather and holiday indicators through a separate branch rather than concatenating them with the history. Yang (2022) models outside drivers as changing the rate of change of hidden state, which amounts to learning impulse responses. Our lag kernel is the discrete, interpretable version of that idea. Lan et al. (DSTAGNN) learn links between nodes that respond to the same shocks even when they are far apart geographically. This motivates comparing a learned influence matrix $A$ against simple geographic adjacency.

Two recent systems constrain our design more than they supply it. Aura combines static metadata tokens, cross-attention over sensor series, and a language model that routes inputs to expert sub-networks. CastFSR lets a language-model agent adjust a numerical forecast using retrieved text context and then checks the adjusted forecast against domain rules. Both were built for high-frequency industrial and energy data with abundant training signal. Neither fits a few hundred stage transitions, so we use them only as baselines.

On the governance side, the work motivates the question rather than supplying a method. Bommasani et al. propose "if-then" protocols that trigger rules when capability milestones are crossed. Whether such pre-commitments are needed depends on how slow the ordinary reactive response is, and that is exactly what we measure. The AGILE Index documents a 12-fold rise in AI risk incidents in 2023, but only at annual, national resolution. Luna et al. describe statutes as decaying snapshots but give no rate. Our kernel supplies the missing timescale on the response side.

Research Program

First, we rebuild the Khan et al. bill set with week-level histories of each bill's stage changes from the legislative tracking records the dataset was drawn from. Where feasible, we extend it with keyword variants to reduce the narrow-query problem that paper acknowledges.

Second, we build weekly outside-signal series. These include counts of publicly reported AI incidents, dates of major model releases, and enactment events in other states. Where possible, a topic tag lets a bill on deepfakes respond to deepfake incidents specifically. These series must be assembled by us, since the file names no ready-made incident time series. We fix the construction rules before looking at outcomes.

Third, we fit the hazard model with smooth, low-dimensional kernels, using a few basis functions over lags of up to 26 weeks so that the small sample is not over-parameterized. Uncertainty comes from bootstrapping over states. As a secondary model, a gradient-boosted hazard with the same inputs checks whether the linear kernel misses important structure.

Baselines

The naive reference point has two parts: a stage-specific constant hazard plus legislative-calendar effects, and a persistence forecast that each state's introduction rate next quarter equals its last. Any claimed signal must beat this.

The strongest existing method is the Khan et al. two-stage hurdle model. We refit it on identical bills and the identical time split, turning its per-bill probabilities into weekly hazards for comparison, and we add a year trend so it is not unfairly handicapped. The file reports no predictive-accuracy figure for it (no reported result found on Lacuna for Khan et al. out-of-sample performance), so we report our own refit.

The third baseline is a CastFSR-style language-model adjuster. The Khan-refit forecast serves as the prior, and a language model reads bill text and recent news to make a bounded adjustment, with the same tuning budget of validation-set prompt selection. To prevent the model from simply recalling what happened, it is evaluated only on bills whose outcomes postdate the model's training cutoff. An Aura-style fusion network is reported for completeness at matched parameter tuning.

Ablations remove, one at a time, the outside-signal kernels, the neighbor-diffusion term, and topic matching. A placebo version shifts each outside series by a random offset of six to twelve months. Beating the Khan refit means a lower integrated Brier score (the mean squared error of predicted probabilities over time) and higher time-dependent concordance on the held-out period, with bootstrap intervals that exclude zero. It also requires that the placebo recover none of the gain.

Why not just…?

Why not just add a year dummy to the hurdle model? AI bill volume has grown steadily, so a trend could absorb most of the apparent effect of incidents. This is a real threat. The placebo-shifted signals settle it: if randomly offset incident series predict as well as the correctly timed ones, the "lag" is only trend, and we will report that as a negative result.

Why not just let a language model read the bills and predict their fate? Language models carry strong background knowledge about politics, and with 18 safety bills that knowledge could beat any fitted model. We run it as a baseline, with post-cutoff evaluation to prevent recall. If it wins, the lag kernels still offer something it does not: an explicit estimated delay with uncertainty. We would then state plainly that forecasting accuracy is not our advantage.

Why not just use a standard Cox survival model with time-varying covariates? That is essentially what our model is, and we say so. The contribution is not a new estimator. It is the time-stamped outside-signal series, the topic matching, the between-state diffusion term, and a forward-in-time forecasting test. These together turn "pacing problem" rhetoric into a delay that can be measured.

Decisive Checks

We train on 2017–2023 and test on 2024–2026. Three results would decide the proposal.

First, the full model must lower held-out Brier score relative to the Khan refit, and the placebo must erase the improvement.

Second, the estimated kernel peak for committee-to-floor transitions must be stable across bootstrap samples, with an interval narrow enough to be meaningful, for example within ±6 weeks.

Third, a learned influence matrix $A$ must beat geographic adjacency, which would indicate that policy diffuses along non-geographic ties, as DSTAGNN found for traffic nodes.

If topic-matched kernels are slower for safety bills than for consumer-protection bills, that is direct evidence for Bommasani et al.'s argument that reactive legislation cannot keep pace in high-stakes areas.