Leading Johnny to Water: Designing for Usability and Trust

Erinn Atwater, Cecylia Bocovich, Urs Hengartner, Ed Lank, Ian GoldbergSOUPS 2015 · OpenReview

TL;DR

Despite the availability of end-to-end encryption (E2E) for years, widespread adoption remains elusive. This paper investigates why, finding that while integrated E2E email clients are significantly preferred for their usability, user trust is not primarily driven by transparency or manual interaction with ciphertext, but rather by perceptions of developer trustworthiness and the convenience of integration.

What Problem Does This Paper Address?

The paper tackles the persistent problem of low adoption rates for end-to-end (E2E) encrypted email. While the technology has existed for decades, users are not widely using it. Previous research, notably "Why Johnny Can't Encrypt," has highlighted usability as a major barrier. However, this paper delves deeper, questioning whether usability alone is the sole culprit or if other factors like user trust, awareness, and the perceived complexity of encryption play a larger role. The authors aim to disentangle the effects of integration (how E2E fits into existing workflows) and transparency (how much the user sees of the encryption process) on both usability and trust.

Methods

The researchers designed and evaluated three E2E email encryption tools:

  1. Transparent Integrated Tool: A Chrome browser extension that automatically encrypts/decrypts emails within a webmail client (Gmail) and displays the ciphertext.
  2. Opaque Integrated Tool: Similar to the transparent version, but hides the ciphertext behind a user-friendly overlay.
  3. Transparent Standalone Tool: A separate desktop application requiring users to manually copy/paste messages for encryption/decryption, also displaying ciphertext.

They conducted a mixed-methods study involving focus groups, pilot studies, and a main user study with 36 participants. The study used the System Usability Scale (SUS) for quantitative usability assessment and collected qualitative data through questionnaires and interviews to gauge user preference, trust, and understanding.

Datasets

Not discussed in the paper. The study used custom-built tools and a controlled experimental setup rather than pre-existing datasets for training or evaluation.

Key Insights and Ideas

  • Integration Trumps Transparency for Usability: Users overwhelmingly prefer integrated solutions that fit seamlessly into their existing email clients (like Gmail) over standalone applications. This preference is driven by convenience and reduced cognitive load, not by the ability to see ciphertext.
  • Trust is Not Solely About Seeing Ciphertext: Contrary to some prior hypotheses, users' trust in E2E encryption tools is not significantly influenced by whether they can see the raw ciphertext. The ability to manually interact with encryption processes doesn't automatically equate to higher trust.
  • Developer Reputation and Perceived Security Drive Trust: Trust is more strongly linked to the perceived trustworthiness of the software developer and the perceived security of the platform (e.g., desktop apps vs. browser extensions). Some participants viewed desktop applications as inherently more secure because they weren't perceived as "leaking" data to the browser or provider.
  • Usability is Key for Adoption, But Trust is a Hurdle: While integrated tools are highly usable, the paper acknowledges that trust remains a significant factor that can hinder adoption, especially if users are unsure about the developer or the underlying security.
  • Automatic Key Management is Welcome: Users appreciate automated key management features that simplify the process of encrypting messages to new contacts, even if it means they don't fully understand the underlying mechanics.

Key Results

  • Usability: Both integrated and standalone tools achieved comparable "good" SUS scores (75±14 and 74±13 respectively) when participants used them first. However, when used second, the standalone tool's SUS score dropped significantly (52±21), indicating a negative learning effect or comparison bias. Overall, the integrated client had a significantly higher SUS score (73±15) compared to the standalone (63±20).
  • Preference: 81% of participants preferred the integrated client over the standalone tool, citing convenience and reduced workflow disruption as primary reasons.
  • Trust: The majority of participants (69%) did not express a trust preference between the two systems. Of those who did, 28% trusted the standalone tool more, often due to its perceived separation from the browser. Only a small fraction expressed concerns about transparency.

Limitations and Future Work

  • Participant Pool Bias: The study primarily recruited students, which may not be representative of the general public's technical expertise or trust perceptions.
  • Minor Design Differences: Despite efforts to standardize, subtle differences between the tools might have influenced trust perceptions.
  • Browser Extension Warnings: The warnings displayed during browser extension installation may have negatively impacted trust in the integrated tool for some participants.

Future work could explore:

  • Designing integrated tools that explicitly convey security assurances (e.g., local key storage) to combat the "browser extension distrust."
  • Investigating how to build user trust through developer reputation, transparent security practices, and user reviews.
  • Further exploring the impact of opacity on user trust, as one participant found opacity preferable for usability.
  • Developing more robust solutions for key management that are both secure and user-friendly.

Summary updated

BibTeX
@inproceedings{atwater2015leading,
  author = {Erinn Atwater and Cecylia Bocovich and Urs Hengartner and Ed Lank and Ian Goldberg},
  title = {{Leading Johnny to Water: Designing for Usability and Trust}},
  booktitle = {SOUPS},
  year = {2015},
  url = {https://openreview.net/forum?id=a9UzlHPLKx},
}
Figure