Interactions between Health Searchers and Search Engines

George Philipp, Ryen W. White

TL;DR

This paper explores the complex feedback loop between health-anxious searchers and search engines, a phenomenon often termed "cyberchondria." By analyzing longitudinal search logs, the authors demonstrate that subtle nuances in query wording reveal significant underlying user concern, which search engines may inadvertently amplify through personalization, potentially reinforcing a cycle of medical anxiety.

What Problem Does This Paper Address?

The Web is a primary tool for self-diagnosis, but it can be a double-edged sword. "Cyberchondria" occurs when common, benign symptoms (like a headache) lead users to believe they have rare, serious illnesses (like a brain tumor) due to the information they encounter online.

The authors set out to understand the mechanics of this interaction. Specifically, they investigate how a user’s subjective concern shapes their queries, how search engines interpret those queries to compile Search Engine Result Pages (SERPs), and how the resulting information influences the user's subsequent clicks and real-world health-seeking behavior. The goal is to provide a roadmap for search engines to better support exploratory health searches without fueling unnecessary anxiety.

Methods

The researchers modeled the search process as a five-stage pipeline:

  1. Prior Behavior: Medical search activity leading up to a symptom search.
  2. Query Formulation: The specific wording chosen for a "landmark" query (the first exploratory query for a symptom).
  3. SERP Content: The level of "concern" (likelihood of leading to serious condition searches) in the results returned.
  4. Click Behavior: Which results the user chose to engage with.
  5. Future Behavior: Search activity and "Healthcare Utilization Intent" (HUI) following the search.

To quantify "concern," they utilized a pre-trained logistic regression classifier that scores web pages based on whether they tend to trigger searches for serious vs. benign conditions. They also developed a taxonomy of 19 high-level query features (e.g., intensity, duration, pronouns) to categorize how users phrase their medical concerns.

Datasets

The study utilized anonymized search logs from a major commercial search engine, involving:

  • 190,000 consenting users.
  • Six months of data (September 2012 to February 2013).
  • Focus on "headache" queries: Chosen because headaches are common and have a wide range of explanations, from caffeine withdrawal to aneurysms.
  • 50,000 unique queries and over 190,000 query instances.

Key Insights and Ideas

  • Nuance is Data: Seemingly identical queries like "daily headache" and "headache everyday" are not interchangeable. They correlate with different levels of prior medical search intensity, suggesting that word choice is a powerful signal of a user's psychological state.
  • The Personalization "Filter Bubble": Even when entering the same query, users with a history of searching for serious conditions receive slightly more "concerning" SERPs. This suggests that search engine personalization might inadvertently reinforce health anxieties.
  • Selective Exposure: Users don't just receive biased results; they actively seek them out. Medically concerned users are significantly more likely to click on alarming results, even when those results are ranked lower on the page.
  • The HUI Paradox: One of the most surprising findings was that viewing highly concerning pages led to a short-term reduction in queries related to seeking professional medical help (HUI). The authors speculate that alarming information might temporarily overwhelm or "freeze" a user rather than prompting immediate action.
Figure 1: Association between query formulation and search behavior features. From left to right: Red: pastmedical (aggrega- tion window: 90 days); Blue: pastmedical (window: 1 hour); Brown: futuremedical (window: 1 hour); Black: futuremedical (window: 90 days). All bars show the relative change of medical search intensity of users whose landmark query has a certain formulation relative to the global mean. Error bars are 95% confidence intervals.
Figure 1 illustrates how different query formulations (like adding "intensity" or "duration") correlate with varying levels of past and future medical search activity.

Key Results

  • Search Engine Response: Search engines generally do a good job of distinguishing between benign and serious symptoms in queries. However, there is significant "noise"—for instance, a user typing "terrible headache" receives a more concerning SERP than one typing "bad headache," despite the semantic similarity.
  • Prior Search Impact: Users who received the 10% most concerning SERPs for a given query had searched for 18% more serious conditions in the preceding 24 hours compared to those receiving the 10% least concerning SERPs (see Figure 3.2).
  • Reinforcement Cycle: Users who select more concerning pages are significantly more likely to have searched for 19% more serious conditions in the 30 days prior to the landmark query (see Figure 3.4).
  • Behavioral Shift: Users receiving especially concerning SERPs performed 20% fewer HUI queries (e.g., searching for a doctor or emergency care) in the hour following the search compared to their peers (see Figure 3.7).
Figure 3: Association between user search features and features related to the landmark query. Analysis was conducted as described in section 3.3. In each case, we show the ratio between the search behavior feature value across all upper buckets and across all lower buckets. Hence, positive values imply a positive association between features and vice versa. The p-value is noted next to the bar. We do not show results that were not significant, except in Figure 3.7, were only futurehuiclicked was significant. The effective sample size refers to the total number of individuals / datapoints included in the analysis. This varies for three reasons. (1) Some features are unavailable for some users. For example, in Figure 3.6, we cannot include users that did not click on the SERP. (2) In some Figures, we only include a specific subset of users. For example, in Figure 3.2, we only include users receiving the 10% most / least concerning SERPs for each query. (3) We have to subsample for statistical reasons (see section 3.3).

Limitations and Future Work

The authors acknowledge several limitations:

  • Single Symptom Focus: The study was limited to headaches; results might differ for other symptoms.
  • Proxy Metrics: Search logs provide a window into behavior but cannot capture a user's actual medical state or their internal motivations.
  • Classifier Accuracy: The "SERPConcern" metric, while useful, is an automated estimate and may contain errors.

Future Work includes expanding the analysis to multiple symptoms, refining how search engines detect health anxiety, and finding a balance in result presentation that prevents unnecessary panic while ensuring users don't delay critical medical treatment when they are actually sick.

Summary updated

Abstract

The Web is an important resource for understanding and diagnosing medical conditions. Based on exposure to online content, people may develop undue health concerns, believing that common and benign symptoms are explained by serious illnesses. In this paper, we investigate potential strategies to mine queries and searcher histories for clues that could help search engines choose the most appropriate information to present in response to exploratory medical queries. To do this, we performed a longitudinal study of health search behavior using the logs of a popular search engine. We found that query variations which might appear innocuous (e.g. "bad headache" vs "severe headache") may hold valuable information about the searcher which could be used by search engines to improve performance. Furthermore, we investigated how medically concerned users respond differently to search engine result pages (SERPs) and find that their disposition for clicking on concerning pages is pronounced, potentially leading to a self-reinforcement of concern. Finally, we studied to which degree variations in the SERP impact future search and real-world health-seeking behavior and obtained some surprising results (e.g., viewing concerning pages may lead to a short-term reduction of real-world health seeking).
BibTeX
@inproceedings{philipp2014interactions,
  author = {George Philipp and Ryen W. White},
  title = {{Interactions between Health Searchers and Search Engines}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2014},
  doi = {10.1145/2600428.2609602},
}
Figure