Search-based disease trackers work well where they are needed least. A tracker of this kind estimates current case counts from how often people search for a disease. Models such as ARGO (autoregression combined with Google search data) reach very high correlation with clinical counts for seasonal dengue and influenza, because they can lean on years of past case counts. That history lets them learn which search terms track infection and which merely track news. When a pathogen is new, or a disease reaches a region where it was never endemic, that history does not exist. Search volume in that setting is driven mostly by public alarm rather than infection. ARGO itself was beaten by a naive "same as last week" model in Taiwan, where dengue is sporadic. Recommender-systems researchers call this situation the cold-start problem: you must make predictions for an entity (a new user, a new item, here a new disease) that has few or no observed outcomes.
We propose to treat the emerging-outbreak case as a cold-start problem whose main obstacle is media-induced search bias. The key observation is that the way the public's searching responds to news coverage is probably more transferable across diseases than the relationship between searches and infections. A celebrity diagnosis or an alarming headline produces a surge of curiosity searches, and that surge has a shape: a sharp rise, then decay, then fatigue as coverage saturates. That shape can be learned from diseases where clinical counts are known. A new disease then needs only a small number of disease-specific quantities, and those can be drawn from a prior learned across warm diseases and updated from a handful of early clinical reports.
Core Claim
Search volume for a disease can be split into three parts. One is driven by infection. One is driven by news coverage through a response curve that is shared across diseases. The third is a baseline level of curiosity. If the news-response curve and the prior over the infection scale are learned on diseases with clinical history, a new disease or region can be nowcast from search and news volume alone. Concretely, for disease $d$ at week $t$, let $s_{d,t}$ be log search volume, $y_{d,t}$ the true incidence, $n_{d,t}$ log news volume about the disease, and $N_{d,t}$ the cumulative news exposure so far. We model
$$ s_{d,t} = c_d + \beta_d \log(1 + y_{d,t}) + \sum_{k=0}^{K} h_k(N_{d,t-k})\, n_{d,t-k} + \varepsilon_{d,t}, $$
Here $c_d$ is baseline curiosity, $\beta_d$ is how strongly infection drives searching, and $h_k(\cdot)$ is the news-response kernel at lag $k$. The kernel weakens as cumulative exposure $N$ grows, which encodes the saturation pattern described below. The parameters $c_d$ and $\beta_d$ get a hierarchical prior conditioned on observable disease features: symptom specificity of the search terms, whether the disease is endemic or sporadic, and prevalence class. The kernel $h$ is shared, with only small disease-level deviations allowed. Nowcasting a cold disease means inverting this equation for $y_{d,t}$ under the posterior. That posterior equals the prior when no clinical data exist and sharpens as early reports arrive.
The saturation term is motivated directly by Tracking COVID-19 using taste and smell loss Google searches is not a reliable strategy. That paper showed that taste and smell loss searches rose with media attention and then decoupled from later infection waves once curiosity was exhausted. A response curve that ignores fatigue would misread every later wave.
What is new is the unit of transfer. Existing trackers transfer nothing across diseases. Existing cold-start recommenders transfer preference structure, not a model of exogenous attention shocks. We transfer the attention model and keep the infection link disease-specific.
Relation to Prior Work
Three lines of epidemiological work motivate the decomposition, and each assumes a clinical history we will not have.
Predicting Epidemic Tendency through Search Behavior Analysis added news volume to a model of hand-foot-and-mouth disease from search logs. The fitted news coefficient came out negative, effectively subtracting panic searches, and correlation rose from 0.735 (queries alone) to 0.891. That paper fits one disease with its own history and uses a single static coefficient. We generalize its news term into a lagged, saturating curve shared across diseases.
Estimating Rare Disease Incidences With Large-scale Internet Search Data separated health-seeking from news-driven search sessions and cut relative error rate from 0.511 to 0.365 across rare diseases, including the ALS spike after Stephen Hawking's death. Its second stage, however, regresses on registry incidence from more than 300 hospitals, which an emerging disease lacks.
The ARGO family includes Advances in using Internet searches to track dengue, Use Internet search data to accurately track state level influenza epidemics, Improved Real-Time Influenza Surveillance: Using Internet Search Data in Eight Latin American Countries, and COVID-19 Hospitalizations Forecasts Using Internet Search Data. These models remove media-driven terms with sparse (Lasso) regression over rolling windows of clinical data and borrow strength across states. All of them stay within one disease. The Latin American study notes that reporting gaps in Bolivia and Uruguay weaken calibration, and the dengue paper documents the Taiwan failure. These are the failure cases we target.
The recommender literature supplies method rather than motivation.
MAMO: Memory-Augmented Meta-Optimization for Cold-start Recommendation showed that a single global initialization adapts poorly when entities differ from the population average. Its profile-conditioned memories reduced mean absolute error in the cold-user, cold-item setting on MovieLens-1M from 0.9299 (MeLU) to 0.8894, adapting from 5 to 15 interactions. Our feature-conditioned prior over $(c_d, \beta_d)$ plays the role of MAMO's profile memory, with disease archetypes in place of user demographics.
DisCo is a constraint on our design. It found that forcing two signal spaces into alignment collapses information unique to each. We therefore do not force a shared infection link $\beta$. Only the attention component is shared, and the experiments test whether even that sharing is justified.
DisAlign and DCDIR show that cross-domain transfer rescues cold targets in recommendation. Both assume the source domain's behavior reflects stable preferences. Our model, by contrast, explicitly separates a shock component that corrupts source and target alike.
Research Program
The first component is a multi-disease, multi-region panel assembled from public sources used in the cited papers. It covers weekly Google Trends volume for disease and symptom terms, together with clinical series:
- dengue counts for Brazil, Mexico, Thailand, Singapore and Taiwan, following the ARGO dengue study;
- WHO FluNet specimen counts for the eight Latin American countries;
- CDC ILINet state-level influenza-like illness;
- HHS COVID-19 hospital admissions.
We add a news-volume series per disease and region, built from a news archive with the same keyword seeds. The Sogou rare-disease logs and registry used by Li et al. are proprietary. We will seek them through collaboration. If access is granted, they give the cleanest leave-one-disease-out test, since they include fifteen diseases and a documented media shock.
The second component is the model above, fit by hierarchical Bayesian inference. Its parameters are:
- a shared saturating kernel, parameterized as a small number of decaying basis functions whose amplitudes shrink with cumulative exposure;
- disease-level $(c_d, \beta_d)$ with a prior regressed on disease features;
- a latent incidence process with a weak smoothness prior, so the model has some temporal anchoring even with no clinical lags.
Few-shot updating means conditioning on the first $m$ clinical observations, with $m$ set to 0, 2, 4 or 8 weeks. Once enough history exists, we hand off to a standard ARGO model. We expect our advantage to vanish at that point, and we will measure where the crossover falls.
The third component is a set of controlled cold-start simulations. "Leave one disease out" holds out an entire disease. "Leave one region out" holds out, for example, Taiwan dengue or Bolivia influenza. "Truncated onset" replays the first months of COVID-19 in 2020, when no hospitalization history existed. In every case the held-out entity contributes only search and news data, plus its first $m$ clinical points.
Baselines
The do-nothing reference is the naive persistence model, which predicts last reported value as this week's value and is defined whenever at least one clinical point exists. With zero clinical points, the reference is a prior-mean predictor using the average incidence trajectory of warm diseases with the same features. Persistence is a serious competitor: it beat ARGO in Taiwan according to Advances in using Internet searches to track dengue. That paper's Taiwan performance numbers are not in the file, so no reported result was found on Lacuna for Taiwan persistence; we will recompute it. Beating it means lower mean absolute error during outbreak weeks without worse error in quiet weeks.
The strongest existing method is ARGO fitted on whatever $m$ weeks of clinical data are available, using the original recipe of log search terms, sparse regression and autoregressive lags. On full history it reports Pearson correlations of 0.971 for Brazil dengue, according to Advances in using Internet searches to track dengue, and 0.957 for Brazil influenza against 0.891 for plain autoregression, according to Improved Real-Time Influenza Surveillance. These full-history numbers are a ceiling we do not expect to reach in the cold regime. We will reproduce ARGO on full history first, to confirm our pipeline matches these figures, before truncating. All methods see identical search terms, news series and clinical points, with hyperparameters tuned by the same rolling-origin validation on warm entities only.
A stronger transfer baseline is pooled cross-disease ARGO. It fits one sparse regression on standardized search and lag features across all warm diseases and applies it directly to the cold one. A second transfer baseline is the same pooled model with Xu et al.'s linear news term added. On the rare-disease data, if available, the intent classifier from Estimating Rare Disease Incidences With Large-scale Internet Search Data is trained on other diseases. Its health-seeking session counts are then calibrated with the same $m$ points. Its full-history relative error rate of 0.365 serves as the upper reference.
Two ablations isolate the key ingredient. The first removes the news kernel entirely, leaving only the hierarchical prior over infection scale, which amounts to MAMO-style meta-learning without an attention model. The second keeps the kernel but removes saturation. Beating the pooled baselines and the first ablation in the $m \le 4$ regime is the result that would support the claim.
Why not just…?
Why not just use persistence and wait for clinical reports? For sustained, seasonal epidemics, the cited ARGO papers show persistence lags inflection points; the COVID-19 hospitalization paper reports 27% lower RMSE than Forecast Hub models from search signals with 4- to 7-day leads. In sporadic settings, though, persistence genuinely wins. We treat that as a real threat. The Taiwan and Bolivia leave-out experiments are designed so that persistence can beat us, and we will report it if it does.
Why not just pool all diseases into one ARGO regression? This is the most serious alternative, because pooling already transfers. It assumes one shared mapping from search to incidence. That is precisely the assumption DisCo warns against, and it is contradicted by the spread in search-to-case relationships across the cited studies. Pooling also cannot represent news-driven excess that saturates. The comparison against pooled ARGO, with and without a linear news term, settles this directly. If pooling with a linear news term matches our model, we will conclude that the saturating shared kernel adds nothing and simplify accordingly.
Why not just use clinical search terms that curious laypeople do not type? The ALS spike after Hawking's death hit a technical disease name. Analyzing Search Behavior of Healthcare Professionals for Drug Safety Surveillance found that even clinicians' queries spiked after FDA alerts and journal publications. Term selection reduces the media component but does not remove it. We include a specialist-terms-only variant of the pooled baseline to measure how much bias remains.
Why not just use pharmacy sales? Predicting seasonal influenza using supermarket retail records argues that purchases are less sensitive to media cycles. Such data are proprietary, though, and absent for diseases without over-the-counter treatments, which includes most emerging and rare diseases. Our model could add sales as another infection-driven channel where it exists, but we do not depend on it.
Proposed Experiments and Decisive Checks
The first check comes before any nowcasting: does the attention kernel actually transfer? We fit the kernel on all diseases but one. On the held-out disease, during known media events and with true incidence observed, we then predict the search excess left after removing the infection term. Key events include Hawking ALS if Sogou data are available, taste and smell loss in spring 2020, and dengue alarm in Taiwan. If the transferred kernel explains no more held-out excess than a kernel fit on shuffled news series, the central premise fails and the project reduces to pooled transfer.
The main experiment reports mean absolute error, RMSE and Pearson correlation as a function of $m$, separately for outbreak and non-outbreak weeks. It covers every leave-out scenario. Uncertainty comes from block bootstrap over weeks, and per-entity win rates are reported against persistence and pooled ARGO. We expect the largest gains at $m = 0$ to $4$ and convergence to ARGO by roughly a season of data. The point where the curves cross is itself a useful finding for surveillance agencies.
A confounding check addresses the model's main weakness. News coverage rises partly because outbreaks are real, so the kernel can absorb true signal. We test this by comparing kernel estimates fit on media events with little associated incidence, such as celebrity cases and regulatory alerts, against kernels fit on outbreak-driven coverage. In the early-2020 COVID replay, we report how much of the true hospitalization rise the model wrongly attributes to attention. If misattribution exceeds the error saved, the model needs news sources tagged as outbreak-unrelated, and we will say so rather than tune it away.
References
- Tracking COVID-19 using taste and smell loss Google searches is not a reliable strategy
- Predicting Epidemic Tendency through Search Behavior Analysis
- Estimating Rare Disease Incidences With Large-scale Internet Search Data: Development and Evaluation of a Two-step Machine Learning Method
- Advances in using Internet searches to track dengue
- Use Internet search data to accurately track state level influenza epidemics
- Improved Real-Time Influenza Surveillance: Using Internet Search Data in Eight Latin American Countries
- COVID-19 hospitalizations forecasts using internet search data
- MAMO: Memory-Augmented Meta-Optimization for Cold-start Recommendation
- DisCo: Towards Harmonious Disentanglement and Collaboration between Tabular and Semantic Space for Recommendation
- Leveraging Distribution Alignment via Stein Path for Cross-Domain Cold-Start Recommendation
- DCDIR: A Deep Cross-Domain Recommendation System for Cold Start Users in Insurance Domain
- Analyzing Search Behavior of Healthcare Professionals for Drug Safety Surveillance
- Predicting seasonal influenza using supermarket retail records