Designing for data protection

Carmela Troncoso, Wouter LueksCambridge 2024 · DOI · OpenAlex

Abstract

Humanitarian Organizations assist the most vulnerable populations in extremely challenging circumstances. For reasons of efficiency, accountability, and out of a desire to help as many people as possible, Humanitarian Organizations increasingly rely on digital technology in their programmes. The livelihood and safety of vulnerable populations often relies on the assistance provided by these organizations. As a result, individuals have very little agency in whether to accept the assistance and whether to participate in these digital systems if they wish to accept the assistance. Digital systems bring data protection and privacy risks. Especially for vulnerable populations, these risks might be significant. Therefore, humanitarian organizations have an obligation not just to safeguard individuals’ livelihood in the short term, but also to uphold data protection as well as privacy rights and the dignity of the people they help.

The Challenge of Digital Aid

Humanitarian Organizations (HOs) are undergoing a rapid digital transformation to deliver aid more effectively. However, this shift introduces significant risks for the populations they serve—often refugees or individuals in conflict zones. These users typically lack the agency to "opt-out" of digital systems when those systems are the sole gateway to essential resources like food or medical care.

Humanitarian aid increasingly relies on digital systems, creating data trails for vulnerable populations.

A critical issue identified is that regulatory compliance, such as GDPR, often acts as a "paper shield." It relies on organizational promises rather than technical constraints. In high-stakes environments, "purpose creep"—the repurposing of aid data for surveillance or law enforcement—remains a constant threat. This paper argues for a shift toward systems that are privacy-preserving by design, moving beyond legal checklists to technical enforcement.

A Methodology for Technical Purpose Limitation

The proposed Privacy Engineering Methodology centers on Technical Purpose Limitation. Unlike traditional data minimization, which focuses on deleting unnecessary data after collection, this approach ensures that the system is architecturally incapable of performing actions outside its narrow, stated goals.

The risk mitigation framework focuses on identifying precise system goals and enforcing them through technical design.

The methodology involves several key stages:

  • Narrow Purpose Definition: Shifting from broad objectives (e.g., "contact tracing") to specific technical requirements (e.g., "notifying contacts of an infection").
  • Dual-Directional Risk Identification: A bottom-up analysis of data leakages combined with a top-down assessment of potential societal harms, such as the stigmatization of specific neighborhoods.
  • Decentralization: Prioritizing architectures where sensitive data remains on user devices rather than in central databases, thereby removing the "honeypot" risk.

Key Insights: Beyond the Compliance Trap

Traditional data management follows a "funnel" model: collect data and then apply policies to restrict its use. This paper advocates for a "pyramid" approach, where only the data strictly necessary for the technical purpose is ever accessible to external entities.

A key insight is that Purpose Creep is a design flaw. For instance, when the German Luca app was used by police to track witnesses, it represented a failure of the system's architecture to prevent central data access. Furthermore, the paper notes that data minimization is relative; the goal is not necessarily to reduce the total volume of data processed, but to optimize its distribution to eliminate central points of failure.

Comparative Results: DP3T vs. Centralized Models

The paper evaluates these principles through a comparison of contact-tracing protocols:

  1. DP3T (Decentralized): This protocol successfully implements technical purpose limitation. By handling only random, rotating identifiers on a central server, it prevents the reconstruction of social graphs or user identification.
  2. Centralized (NeedToKnow): This model fails technical purpose limitation. Because the server links identifiers to specific users, it can be easily repurposed for mass surveillance.

While decentralized designs may carry slightly higher "inherent risks" (e.g., a user identifying a specific contact), they drastically reduce "implementation risks," such as large-scale data breaches or state-sponsored surveillance.

Challenges and Future Directions

Implementing privacy engineering in the real world faces several hurdles. Modern "agile" software development often favors flexibility, which conflicts with the "frozen," narrow purposes required for privacy-by-design. Additionally, HOs often rely on third-party libraries and services that act as "black boxes," collecting data outside the organization's control.

Future efforts should focus on bridging the expertise gap within HOs, providing them with the tools to conduct rigorous risk-benefit analyses when choosing between convenient off-the-shelf solutions and custom, privacy-preserving architectures.

Summary updated

Abstract

Humanitarian Organizations assist the most vulnerable populations in extremely challenging circumstances. For reasons of efficiency, accountability, and out of a desire to help as many people as possible, Humanitarian Organizations increasingly rely on digital technology in their programmes. The livelihood and safety of vulnerable populations often relies on the assistance provided by these organizations. As a result, individuals have very little agency in whether to accept the assistance and whether to participate in these digital systems if they wish to accept the assistance. Digital systems bring data protection and privacy risks. Especially for vulnerable populations, these risks might be significant. Therefore, humanitarian organizations have an obligation not just to safeguard individuals’ livelihood in the short term, but also to uphold data protection as well as privacy rights and the dignity of the people they help.
BibTeX
@inproceedings{troncoso2024designing,
  author = {Carmela Troncoso and Wouter Lueks},
  title = {{Designing for data protection}},
  booktitle = {Cambridge},
  year = {2024},
  doi = {10.1017/9781009414630.011},
}
Figure