Is It the Mood or the Rating Scale? Placing Variables in Latent-State Models of Self-Report

When a phone nudge makes a medical intern report a better mood, two different things could have changed. The intern's mood may actually have improved, or only the way the intern used a 1-to-10 rating scale may have shifted. This note proposes a method that decides, for each candidate variable, which of the two it acts on. We borrow a habit from systems biology. There, a model's hidden dynamics are kept separate from the "observation function" that maps hidden states onto what an instrument reads. We treat a self-report questionnaire as one more instrument, with its own person-specific offset, slope, and ceiling. Objective wearable signals, such as step counts and sleep, serve as a second window onto the same hidden state. A greedy stepwise search then decides, for every candidate variable, whether it enters the hidden dynamics, the reporting function, or neither.

The concrete test case is a published and puzzling result. In a micro-randomized trial of 1,565 interns, push-notification tips raised self-reported mood when baseline mood was low and lowered it when baseline mood was high (Assessing Real-Time Moderation for Developing Adaptive Mobile Health Interventions for Medical Interns: Micro-Randomized Trial). On a bounded 1–10 scale, a pattern like that is exactly what compression near the top of the scale would produce. The same study, however, found the same sign of moderation on objective steps and sleep, which no rating scale can distort. Our model can tell how much of the mood reversal is real and how much is the scale, and the answer changes who should receive the nudge.

Core Claim

A stepwise search that assigns each variable a place can separate true intervention effects from reporting artifacts, using self-reports that nobody can independently verify. The two places are the latent (hidden) dynamics and the self-report observation function. The claim is that this separation is possible on standard mobile-health data whenever at least one objective channel shares the hidden state. It is not possible from self-report alone, because a scale ceiling and a true negative moderation then predict identical data.

The optimizer matters for practical reasons, not as the contribution itself. The model's objective is a sum of squared residuals. That makes Levenberg-Marquardt, a least-squares solver, the natural choice: it blends gradient descent with a Gauss-Newton step through a damping term. The same Jacobian it builds also lets us score every candidate variable-placement from the current fit without refitting.

The Model

Let $x_{i,t}$ be the hidden well-being state of intern $i$ on day $t$, and let $A_{i,t}\in\{0,1\}$ record whether a tip was randomized that day. The hidden dynamics are a damped relaxation toward a personal set point $m_i$, plus a treatment effect whose size may depend on the current state:

$$x_{i,t+1} = x_{i,t} + \alpha\,(m_i - x_{i,t}) + (\beta_0 + \beta_1 x_{i,t})A_{i,t} + \gamma^\top z^{\text{dyn}}_{i,t} + \epsilon_{i,t}.$$

Here $\alpha$ is the relaxation rate, $\beta_1$ is the "true" state-dependent moderation, $z^{\text{dyn}}$ holds the covariates selected into the dynamics, and $\epsilon$ is process noise. Observations come from separate readouts. Self-reported mood is $y_{i,t} = 1 + 9\,\sigma\big(s_i(x_{i,t} - c_i) + \delta^\top z^{\text{obs}}_{i,t}\big) + \eta_{i,t}$. In that expression $\sigma$ is the logistic function, which squashes values onto the 1–10 range and so builds in a ceiling. The terms $s_i$ and $c_i$ are the intern's scale slope and offset, $z^{\text{obs}}$ holds the covariates selected into reporting behaviour (for example, the day of the week or a notification being visible on screen), and $\eta$ is reporting noise. Fitbit steps and sleep are linear readouts of $x$ with their own noise. The steps and sleep readouts are the anchors that stop the self-report scale parameters and the hidden dynamics from trading off against each other.

Deciding where a variable goes is a discrete search over three options per candidate: dynamics, observation function, or excluded. We use bidirectional stepwise selection, which adds a variable and then re-checks earlier choices, as in Hoshikuma et al.. Candidates are scored with the Levenberg-Marquardt quantities already in hand, the gradient $J^\top e$ and the curvature approximation $J^\top J + \mu I$. Here $J$ is the Jacobian of the residuals $e$ with respect to all parameters, including the ones a candidate would add, evaluated at zero, and $\mu$ is the damping term. One damped step gives a score-test approximation to the drop in the objective. Only the top few candidates are then refit fully. Additions are accepted by the Bayesian Information Criterion, which penalizes each added parameter. That is the same stopping rule used for clinical stepwise models in Multifaceted evaluation of central post–stroke pain.

Research Program

The interdisciplinary transfer is specific. In the systems-biology benchmark collection of Benchmark Problems for Dynamic Modeling of Intracellular Processes, every ODE model separates dynamics from observables, and the observables carry scaling, offset, and noise parameters. That paper also shows three things we reuse. Log-parameterization improves the shape of the fitting landscape. Second-order trust-region fitting beats interior-point methods on 18 of 20 problems. Cost per step grows quadratically in parameter count. Behavioral mHealth analyses, by contrast, feed raw ratings into weighted least squares, as the interns trial did. Our job is to bring the observation-function idea, together with its identifiability checks, into behavioral data. Doing so needs psychiatric trialists who can say which covariates plausibly affect reporting rather than feeling. It also needs modelers who can diagnose flat directions of the objective.

The build has three parts. The first is a Levenberg-Marquardt fitter for the hierarchical state-space model above. Person-level parameters are pooled toward population values, and the damping $\mu$ is adapted as usual. The second is the placement search with Jacobian-based screening. The third is an identifiability audit that reports the eigen-spectrum of $J^\top J$ for each selected model, so that a placement is only reported when its direction of the objective is well conditioned. Post-selection inference is a known weakness. The homotopy method of More Powerful and General Selective Inference for Stepwise Feature Selection using Homotopy Method gives valid, tight intervals after stepwise selection, but only for linear models. We therefore use sample splitting by intern: selection runs on half the interns and estimation on the other half. We accept the loss of power this causes, a loss that paper documents for data splitting, rather than claim inference we cannot justify.

Baselines

The naive reference is the published analysis itself. It is a weighted, centered least-squares moderation model that treats the 1–10 mood rating as a true continuous outcome. It reports a mood moderation estimate of −0.052 (P = .001), benefit only below mood 7, and objective moderation estimates of −0.039 for steps and −0.075 for sleep (Assessing Real-Time Moderation…). We re-run it on identical data splits.

The strongest competitor for the specific question of scale artifacts is a psychometric model with no dynamics. It is an ordinal mixed-effects regression with person-specific thresholds, which absorbs scale-use differences directly. No reported result for this model on the Intern Health Study was found on Lacuna. We fit it with the same covariates and the same held-out interns. It also gets the same tuning budget: the same number of candidate specifications evaluated by BIC.

The regularized competitor is a group-lasso version of the placement problem. Each candidate has one coefficient group in the dynamics and one in the observation function, and the penalty is tuned by cross-validation over interns. This is the natural reply to anyone who distrusts greedy search, in the spirit of the elastic-net signature in Associations of Ultra-Processed Food Intake….

Two ablations isolate our key ingredients. The first removes the objective anchors by dropping the steps and sleep readouts. The prediction is that placements then become unidentifiable, which should show up as near-zero eigenvalues of $J^\top J$ and unstable bootstrap choices. The second replaces the Jacobian screening with full refits of every candidate. That run tells us whether the shortcut changes any decisions and how much wall-clock time it saves.

Beating the baselines means two things on held-out interns. The first is lower forecast error for next-day objective sleep and steps given past self-reports. The second is placement decisions that are stable, meaning the same choice in at least 80% of bootstrap resamples. A placement that improves in-sample fit but not objective forecasting counts as a loss.

Why not just…?

Why not just use an ordinal model with person-specific thresholds? This is the real threat, because it already handles ceilings and differences in how people use the scale. What it cannot do is say whether moderation seen in mood is mirrored in the hidden state that also drives sleep and steps. It has no dynamics and no shared state across channels. If its forecasts of held-out objective channels match ours, and its implied treatment thresholds match ours, then the dynamic model adds nothing, and we will report that.

Why not just use lasso or elastic net instead of stepwise search? Shrinkage biases exactly the effect sizes we want to interpret. The placement choice is also structured and exclusive: a variable should enter the dynamics or the observation function, not smear across both. The group-lasso baseline tests this directly. If it recovers the true placements in simulation as often as stepwise search does, we will switch to it.

Why not just fit with Adam or plain gradient descent? For the final fit, a first-order method would probably work, only more slowly. The review in A Comparison of Projection Pursuit and Neural Network Regression Modeling found Gauss-Newton training one to two orders faster than sequential gradient descent. Where Levenberg-Marquardt earns its place is inside the search. There it supplies cheap candidate scores and the curvature matrix used for the identifiability audit. The full-refit ablation, run with a first-order fitter, measures this directly.

Why not just analyze the objective outcomes and drop self-report? Mood has no objective equivalent in this dataset, and it is the outcome clinicians care about. Dropping it answers a different question.

Proposed Experiments

The first experiment is a simulation with known ground truth. We generate intern trajectories from the model with parameter ranges set to match the trial, and vary two things independently. One is true moderation, meaning $\beta_1$ is either nonzero or zero. The other is reporting compression, meaning the ceiling is strong or absent. The decisive check here is a specific case: data with a pure ceiling and no true moderation. The naive least-squares baseline should report spurious negative moderation. Our model should attribute that moderation to the reporting function, and only when the objective anchors are present.

The second experiment is a solver check on the benchmark ODE collection. We add dummy covariates to several benchmark models and confirm that Jacobian-screened stepwise selection makes the same choices as full refits. Convergence rates are compared against the collection's reported trust-region performance (18 of 20 problems converged, averaging 391 ± 19 iterations).

The main experiment runs on the 2018 Intern Health Study data. This depends on collaboration with the study team, since the file lists no public access route. We fit on half the interns and evaluate on the rest. The decisive empirical question is how the −0.052 mood moderation splits between the hidden state and the scale once sleep and steps anchor that state. If most of it remains in the hidden state, the published "benefit only below mood 7" rule stands on firmer ground. If most of it moves to the scale, the rule should be re-derived from the hidden state before anyone builds an adaptive intervention on it.