When a study asks someone "how many days did you take painkillers in your first trimester?", the answer is not a noisy copy of the truth. It is a blurred, rescaled summary of a hidden day-by-day history. Memory fades with time, answers get rounded, and the person's current mood tilts the report. Epidemiology usually treats this recall bias as a nuisance to be avoided, either by discarding retrospective data or by collecting expensive prospective diaries. We propose to treat it as a measurement device instead. The device has unknown but learnable properties, and it is shared across people. Each retrospective answer becomes a linear measurement of a latent timeline, taken through a distorted "recall operator." That operator has a small number of parameters: how fast memory decays with elapsed time, how much rounding occurs, and how strongly current state inflates or deflates the report. Because thousands of respondents pass through the same operator, and their histories are built from a small shared set of temporal patterns, we can estimate the operator and every respondent's timeline together. Only a small anchor subset needs objective or prospective data.
The mechanism comes from compressed sensing with multiple measurement vectors (MMV). This is the problem of recovering many related sparse signals at once when they share which few components are active, a structure called joint or row sparsity. Standard MMV theory treats errors in the measurement process as small, bounded, and random. Recall bias is large, directional, and depends on the respondent's state. Our claim is that it is also highly structured and shared across respondents. That combination is what makes it estimable, and estimating it lets us recover when exposures happened, which no existing recall-correction practice attempts.
Core Claim
Let $X \in \mathbb{R}^{T \times L}$ hold the latent histories of $L$ participants over $T$ time bins (for example, gestational weeks). We write $X = DC$, where $D$ is a dictionary of temporal patterns, such as a nausea bump peaking around week 8 or a late-pregnancy rise, and $C$ is row-sparse: the cohort as a whole uses only a few patterns, with person-specific weights. Participant $l$ answers retrospective questions $y_l$, and a subset also contributes anchor samples $z_l$, which are exact but sparse. The anchors can be biomarker spot checks, wearable-derived values, or a few prospective diary days. We solve
$$ \min_{C,\,\theta}\; \sum_{l=1}^{L} \big\| y_l - A(\theta, s_l)\, D c_l \big\|_2^2 \;+\; \mu \sum_{l \in \mathcal{S}} \big\| z_l - S_l D c_l \big\|_2^2 \;+\; \lambda \|C\|_{2,1}. $$
Here $A(\theta, s_l)$ is the recall operator. Each of its rows integrates the timeline over the window a question asks about. The integration is weighted by a memory-decay kernel over elapsed time and scaled by a gain $g_\theta(s_l)$ that depends on the respondent's current state $s_l$, for example a current depression score. $\theta$ collects the few shared operator parameters. $S_l$ selects the anchored time points for participants in the anchor set $\mathcal{S}$, and $\mu$ weights the anchor fit. $\|C\|_{2,1}$ is the mixed norm that sums the Euclidean length of each row, so it pushes whole temporal patterns in or out of the cohort model together.
The new element is that the perturbation of the measurement process is parameterized, state-dependent, and estimated from the ensemble. Prior MMV work only bounds that perturbation. Our central empirical prediction follows from the setup. Hold the anchor fraction fixed and add participants: timing-recovery error should keep falling, because every new respondent adds information about the shared $\theta$ and the shared pattern support. Methods that correct each person separately should plateau.
Relation to Prior Work
The MMV literature supplies the method. Duarte and Eldar survey structured compressed sensing and argue that solving related signals independently discards cross-signal structure. Eldar et al. cast joint recovery as block-sparse recovery and give stable-recovery guarantees under a block version of the restricted isometry property. Pham et al. show that the $\ell_{2,1}$ penalty with robust losses, solved by single-loop accelerated first-order methods, gains 2–3 dB over independent frame recovery. They also report 36 dB against 13 dB for standard compressed sensing under Cauchy noise. We reuse their solver design for the $C$-step.
The closest theoretical anchor is Ding et al.. They prove that orthogonal matching pursuit for MMV recovers the support even when both the measurements and the sensing matrix are perturbed, provided the perturbations are small and bounded. They also show that joint recovery averages noise across measurement vectors. This paper constrains our design as well as motivating it. Recall bias is not small, so we cannot rely on their bound directly. Our alternating scheme instead aims to shrink the residual operator error after estimating $\theta$ into the regime their analysis covers.
Liu et al. combine sparsity across multiple domains for single-channel EMG. We borrow the idea of fusing heterogeneous measurement types, but our fusion is across measurement rates: coarse retrospective integrals combined with sharp, sparse anchors. The block-unfolded networks of Rauhut et al. are a later option for speed. That paper also warns that such networks degrade on signals that are not sharply sparse, which is one reason we place sparsity on dictionary coefficients rather than on raw daily values.
The recall-bias literature motivates the operator's form. Shengxin Liu et al. document rounding and decay in recalled breastfeeding duration, and M. Zhang et al. find severe decay over ten-year recall of injections; together they motivate the decay kernel and rounding. Wenqing Wu et al., Pfeiffer et al. and Cataldo et al. show that current affective state skews reports of past events, which motivates the state-dependent gain. Lyons shows that people inflate their pre-injury baseline after an injury, which is a sign-flipped gain of the same kind.
The paper that best frames the target problem is Pengfei Guo et al.. Late-pregnancy acetaminophen reports correlate with cord-blood metabolites ($r > 0.70$), but first-trimester reports do not correlate with mid-pregnancy urine, because the compound's half-life is under four hours. That is a timing problem: sparse sharp anchors and coarse recall need a model that connects them. Finally, Nissen et al. and Feng and Narayanan represent the current practice of avoiding recall altogether. They rely on heuristic user filters or passive sensing, and neither inverts the distortion.
Research Program
The first component is the operator model. Each retrospective item is represented as a windowed integral of the latent timeline. The kernel's weight on a bin depends on elapsed time since that bin; we will try exponential and power-law forms. Rounding enters as a quantization step. We use a smooth surrogate during estimation, following the logic of quantized-sample MMV spectral methods (Gridless Variational Line Spectral Estimation with Multiple Measurement Vector from Quantized Samples). Current state enters as a monotone gain with two or three parameters. Keeping $\theta$ low-dimensional is deliberate: identifiability rests on the operator being shared, so we test its sharedness rather than assuming it.
The second component is estimation. We alternate between a joint-sparse coefficient step, solved with FISTA on the $\ell_{2,1}$ objective in the style of Pham et al., and a low-dimensional $\theta$ step solved by gradient methods. The dictionary $D$ is either fixed from domain knowledge (smooth bumps on the gestational-week grid) or learned from the anchor subset alone. That keeps the dictionary fit from being contaminated by the recall distortion it is meant to help undo.
The third component is an identifiability analysis. This is the main risk of the whole project. State-dependent gain is confounded with true exposure whenever state and exposure are genuinely correlated; depressed people may in fact have experienced more adversity. We will characterize, both analytically in a linearized model and by simulation, how many anchored participants are needed to separate the gain from the true signal, and how that number scales with $L$ and with the correlation between state and exposure.
Baselines
The naive reference takes each retrospective report at face value and spreads the reported quantity uniformly over the window the question covers. This is what most analyses implicitly do. Beating it on timing error is necessary but not interesting.
The strongest existing competitor for the correction task is regression calibration: fit a regression of true window exposure on the reported value, plus state covariates, in the anchor subset, and apply it to everyone else. It is standard in epidemiology, uses exactly our anchor data, and will likely match us on correcting total exposure per window. We will give it the identical anchor set, the same covariates, and a comparable hyperparameter search, and we will give it the same dictionary-based timeline model so that it can also output timelines. We count as beating it only if we reduce onset-timing error and between-window misclassification by a margin that holds across resampled anchor sets. Merely matching it on window totals does not count.
A flexible learned competitor is gradient-boosted trees that predict each week's value from all of a participant's reports, their state, and whatever anchors they have. It is trained on the anchor subset with the same tuning budget, and it tests whether an explicit operator model beats a black box at equal data. Predicting cardiovascular risk using machine learning models notes that such models degrade when their inputs are biased self-reports.
The independent-inversion baseline solves the same objective one participant at a time, with an $\ell_1$ penalty on that participant's coefficients and with $\theta$ fit per person. This isolates the value of pooling across participants.
The key ablation is MMV with the $\ell_{2,1}$ penalty but a fixed, undistorted operator: no decay, no rounding, no state gain. It isolates operator learning, the proposal's central ingredient. We also include an oracle that is given the true $\theta$, as a ceiling.
No reported result found on Lacuna for reconstructing exposure timelines from retrospective reports on any of these datasets, so all baseline numbers will be produced here. For the secondary sensing experiment below, the reference is the Random Forest of Feng and Narayanan, which reports macro-F1 of 56.20 for midday stress and 63.41 for end-of-day stress from passive signals.
Why not just…?
Why not just use regression calibration on a validation subsample? This is the real threat, and we concede it on its home ground. If the scientific question needs only total exposure per window, regression calibration is simpler and probably as good. Our claim is narrower: it cannot move mass between time bins, because it maps a report to a corrected report. The first-trimester failure in Guo et al. is exactly a question of when. The head-to-head experiment reports window totals and onset timing separately, so a reader can see whether our gain is confined to timing, as predicted, or is absent altogether.
Why not just replace recall with prospective diaries and wearables? For decades-long exposures such as childhood adversity studied in the UK Biobank (Hao Qin et al.), prospective data do not exist. Even when they do, attrition skews them heavily: back-pain prevalence moves from 44% among all app users to 93% among persistent users in Nissen et al.. Passive sensing alone reaches only modest stress prediction in Feng and Narayanan. Our proposal still uses prospective data, but as sparse anchors instead of as the whole dataset. The anchor-fraction sweep will show how little prospective data is needed.
Why not just invert each person independently? Theory predicts that pooling helps: Duarte and Eldar on shared support, Ding et al. on noise averaging. Per-person inversion also cannot estimate a state gain from one person's handful of answers. It remains an empirical question here, though, and the independent-inversion baseline answers it directly as $L$ grows.
Proposed Experiments
The first experiment is semi-synthetic, on real timelines. We take real weekly symptom trajectories from the Keleya pregnancy app data (Nissen et al.), restricted to persistent users so that the "truth" is dense. We generate retrospective trimester-level reports through recall operators whose decay, rounding, and state gain span the range suggested by the recall literature. We vary the number of participants $L$ from hundreds to tens of thousands, the anchor fraction from 1% to 20%, and the correlation between state and true exposure. Metrics are normalized reconstruction error of the timeline, error in onset and peak week (in weeks), misclassification of exposure across trimesters, error in the recovered $\theta$, and bias in a downstream association estimate relative to the one computed from the true timelines.
The decisive check has two parts. At fixed anchor fraction, our timing error must decrease with $L$ while regression calibration and independent inversion plateau. The fixed-operator ablation must be clearly worse than the full model whenever the true decay is non-trivial. If neither pattern appears, the core claim is wrong.
The second experiment is a real recall test, and it is the check that matters most. Participants who kept daily diaries in a TILES-style protocol (Feng and Narayanan; TILES-2018 Sleep Benchmark Dataset) are asked retrospective count questions about earlier weeks at delays of one, four, and twelve weeks. Their own diaries then serve as ground truth. If existing releases lack such items, we will run a small collection of this form. This tests whether a low-dimensional operator shared across people actually describes real forgetting. We compare cross-validated fit of the shared operator against per-person operators. If heterogeneity between people dominates, the pooling premise fails, and we will report that.
The third, secondary experiment treats wearables as anchors. In the TILES ICU setting, we hide the daily stress assessments, keep only simulated end-of-week retrospective counts plus wearable features, reconstruct daily stress, and compare macro-F1 against the passive-only Random Forest figures of 56.20 and 63.41. Exceeding them would show that a few retrospective answers, once their distortion is modeled, add information that sensors alone miss.
References
- Structured Compressed Sensing: From Theory to Applications
- Robust recovery of signals from a structured union of subspaces
- Efficient Algorithms for Robust Recovery of Images From Compressed Data
- Robustness of orthogonal matching pursuit for multiple measurement vectors in noisy scenario
- Multi-sparse signal recovery for compressive sensing
- Algorithm unfolding for block-sparse and MMV problems with reduced training overhead
- Breastfeeding and myopia: A cross-sectional study of children aged 6–12 years in Tianjin, China
- Alternative risk factors of HCV infection in a rural community in China
- Clinical Features of Patients with Dysthymia in a Large Cohort of Han Chinese Women with Recurrent Major Depression
- Associations between childhood family emotional health, fronto-limbic grey matter volume, and saliva 5mC in young adulthood
- Recalled Parental Bonding Interacts with Oxytocin Receptor Gene Polymorphism in Modulating Anxiety and Avoidance in Adult Relationships
- Measuring the Population Burden of Injuries—Implications for Global and National Estimates: A Multi-centre Prospective UK Longitudinal Study
- Acetaminophen Use in Pregnancy: A Comparison of Self-Reported Intake with Maternal and Newborn Biomarker Measures
- Prevalence and course of pregnancy symptoms using self-reported pregnancy app symptom tracker data
- Understanding Stress, Burnout, and Behavioral Patterns in Medical Residents Using Large-scale Longitudinal Wearable Recordings
- Gridless Variational Line Spectral Estimation with Multiple Measurement Vector from Quantized Samples
- Unlocking the link: predicting cardiovascular disease risk with a focus on airflow obstruction using machine learning
- Brain Structural Underpinnings of the Differential Effects of Threat and Deprivation on Executive Function: A UK Biobank Study
- TILES-2018 Sleep Benchmark Dataset: A Longitudinal Wearable Sleep Data Set of Hospital Workers for Modeling and Understanding Sleep Behaviors