What breach? Measuring online awareness of security incidents by studying real-world browsing behavior

Sruti Bhagavatula, Lujo Bauer, Apu Kapadia

TL;DR

By analyzing the real-world browsing behavior of 303 users over several years, researchers found that only 16% of participants ever read about major security incidents (like the Equifax or Yahoo! breaches) online. Even when users were likely direct victims of a breach, the vast majority remained unaware, suggesting a critical failure in how security risks are communicated to the general public.

What Problem Does This Paper Address?

Security experts often assume that high-profile data breaches serve as "teachable moments" that encourage users to adopt better security habits. However, most research on this topic relies on self-reported data—surveys where people might claim they are aware of a breach to avoid looking uninformed.

The authors of this paper tackle the "awareness gap" by moving beyond what people say they do and looking at what they actually do. They sought to answer three main questions:

  1. How often do people actually encounter information about security incidents during their normal web browsing?
  2. Do people follow up with constructive actions (like changing passwords or freezing credit) after reading about a breach?
  3. What factors—demographics, browsing habits, or the tone of the news—actually influence whether someone engages with security information?

Methods

The researchers utilized a unique longitudinal dataset from the Security Behavior Observatory (SBO). Unlike a controlled lab study, this involved instrumenting the home computers of 303 participants to capture their actual browsing history (URLs, page titles, and timestamps) and password-entry behavior over several years (2014–2018).

To analyze this data, they employed several technical approaches:

  • Keyword Filtering: They identified visits to pages related to six major security incidents (Equifax, Uber, Ashley Madison, Panama Papers, WannaCry, and Yahoo!).
  • Topic Modeling: They used Non-negative Matrix Factorization (NMF) to categorize users' general browsing habits into "leisure" vs. "professional/technical" categories.
  • Sentiment Analysis: Using the NLTK Vader library, they analyzed the tone of the first article a user encountered about a breach to see if "positive" (constructive) vs. "negative" (alarmist) content changed user behavior.
  • Browsing Trajectories: They analyzed "trajectories"—the 20 pages visited before and after a user first encountered breach news—to measure follow-up actions.

Datasets

  • SBO Dataset: Real-world browsing data from 303 Windows users.
  • Password Data: A subset of 233 users provided hashed password data, allowing researchers to identify who actually had accounts with breached services like Yahoo! or Equifax.
  • Follow-up Survey: 109 participants were surveyed to validate how much of their total browsing (including mobile) was captured by the SBO (estimated at ~57%).

Key Insights and Ideas

  • The "Tech-Savvy" Silo: Security news isn't reaching the general public. The strongest predictors for reading about a breach were being older and having programming knowledge. Security information appears to be circulating primarily within technical circles rather than reaching the most vulnerable average users.
  • The Equifax Paradox: The Equifax breach was one of the most publicized and impactful incidents in history. Yet, only 25% of participants who were likely affected (based on their use of credit-monitoring sites) actually read about it. For the Yahoo! breach, that number was a dismal 2%.
  • Tone Dictates Action: This is a major "aha" moment for science communicators. Users were significantly more likely to take follow-up actions if the first article they read had a positive sentiment (i.e., it was constructive and offered advice) rather than just being a report on the damage.
  • The "One and Done" Problem: Most participants (73%) who engaged with breach news only read about one of the six incidents. Awareness doesn't seem to be cumulative; reading about one hack doesn't necessarily make a user more likely to look for news on the next one.

Key Results

  • Low Overall Awareness: Only 16% of participants visited any web page related to the six major incidents studied.
  • Action After Discovery: On the bright side, if a user did click on a breach-related page, 73% of them took at least one follow-up action, such as reading more about the incident or looking up general security advice.
  • Impact of Content Type: Articles that were "PII++" (breaches involving Social Security numbers and financial data, like Equifax) inspired significantly more follow-up activity than breaches involving only passwords or less sensitive data.
  • Demographic Significance: For every standard deviation increase in a user's "proactive awareness" score (from the SeBIS scale), the odds of them reading about an incident increased by a factor of 1.078.

Limitations and Future Work

The authors acknowledge several constraints:

  • Sample Bias: The SBO participants are all Windows users and may be more (or less) security-conscious than the general population by virtue of agreeing to be in a security study.
  • The Mobile Gap: The data only tracks desktop browsing. While their survey suggests this covers the majority of "serious" browsing, it misses news encountered via mobile apps (like Facebook or Twitter).
  • Content Consumption vs. Loading: The researchers can see that a page was loaded, but they cannot prove the participant actually read and understood the content.

Future Work: The authors suggest that news organizations and security professionals need to move away from "fear-based" reporting. Instead, they should focus on constructive, positive-toned communication and explore non-text mediums (like video) to reach non-technical audiences who are currently being left out of the security conversation.

Summary updated

Abstract

Learning about real-world security incidents and data breaches can inform people how their information is vulnerable online and thus encourage safer security behavior. This paper examines 1) how often people read about security incidents online, 2) of those people, whether and to what extent they follow up with an action (e.g., trying to read more about the incident), and 3) what influences the likelihood that they will read about an incident and take some action. Our quantitative study of the real-world internet-browsing behavior of 303 participants finds a low level of awareness. Only 16% of participants visited any web page related to six widely publicized large-scale security incidents; few read about an incident even when it was likely to have affected them. We also found that more severe incidents and articles that constructively spoke about the incident were associated with more action. Our findings highlight two issues: 1) security awareness needs to be increased; and 2) current awareness is so low that expecting users to be aware and take remedial action may not be effective.
BibTeX
@inproceedings{bhagavatula2021what,
  author = {Sruti Bhagavatula and Lujo Bauer and Apu Kapadia},
  title = {{What breach? Measuring online awareness of security incidents by studying real-world browsing behavior}},
  booktitle = {EuroUSEC},
  year = {2021},
  doi = {10.1145/3481357.3481517},
}
Figure