TL;DR
PrivacyScore is a new open-source platform that allows users to benchmark websites for security and privacy features through crowd-sourced data. It aims to increase transparency and incentivize website owners to improve their practices by providing detailed, customizable, and publicly accessible assessments.
What Problem Does This Paper Address?
Website operators make decisions that can expose users to privacy and security risks. While many security scanning tools exist, they often focus on narrow aspects like encrypted connections and are geared towards administrators, making it difficult for end-users to understand a site's overall privacy and security posture. Furthermore, existing tools offer little incentive for website owners to improve, as results are not easily comparable or actionable for the general public. This paper addresses the need for a comprehensive, user-friendly, and incentive-driven platform for evaluating website security and privacy.
Methods
PrivacyScore employs an automated website scanning portal that allows users to benchmark multiple websites. Key features include:
- Crowd-Sourced Benchmarking: Users can create and share lists of websites (e.g., all public schools in a country) for comparison. These lists and their rankings are publicly visible and automatically updated.
- Comprehensive Checks: PrivacyScore goes beyond basic security checks to cover a wider range of potential privacy and security issues, including tracking, third-party services, cookies, browser fingerprinting, and insecure configurations.
- User-Defined Ranking and Analysis: Users have control over the ranking methodology and can define their own criteria or adapt existing ones, allowing for personalized assessments and catering to different audiences (e.g., data protection authorities, privacy advocates).
- Actionable Advice: The platform provides explanations of risks and actionable advice for website operators to improve their scores.
- Open Source and Public API: The platform is open-source, and all collected data is available via a public RESTful API, promoting transparency and enabling further research.
Datasets
The paper does not explicitly mention training datasets. However, it relies on:
- EasyList: A list of known advertisers and trackers used to identify third-party tracking services.
- GeoIP database: Used to determine the geographic location of servers.
- Publicly available data: The platform collects and makes public scan results for various websites, forming a benchmark.
PrivacyScore introduces a new benchmark by allowing users to create custom lists of websites based on shared characteristics, which are then automatically analyzed and ranked.
Key Insights and Ideas
- Transparency as an Incentive: The core idea is that public, comparative rankings of website security and privacy will create an incentive for site owners to improve their practices to avoid negative public perception.
- User Control is Crucial: Empowering users to define their own ranking schemes makes the platform more versatile and relevant to individual privacy preferences and specific regulatory needs.
- Holistic Approach to Privacy: Moving beyond just SSL/TLS checks to include tracking, third-party embeds, and browser fingerprinting provides a more complete picture of a website's privacy impact.
- Dual-Use Nature of Information: The authors acknowledge that the detailed information provided by PrivacyScore could potentially be used by attackers, necessitating careful design of checks and result presentation.
Key Results
The paper describes the design and implementation of PrivacyScore, with a public beta version announced at the ENISA Annual Privacy Forum in June 2017. It details the types of checks implemented (NoTrack, Attacks, EncWeb, EncMail) and the system architecture. While specific quantitative results from large-scale evaluations are not presented in this paper, the focus is on the platform's capabilities and potential impact. The system is designed to be scalable and can be deployed in-house for organizations with specific privacy requirements.
Limitations and Future Work
- Legal and Ethical Considerations: The authors acknowledge that automated scanning without explicit permission might be illegal in some jurisdictions. They currently operate with an "opt-out" policy and suggest that future versions might respect
robots.txtdirectives. - Potential for Misuse: The detailed information provided by the platform could be useful for attackers, requiring careful consideration of what checks are implemented and how results are presented.
- Rate-Limiting: To prevent overloading websites, rate-limiting controls are implemented, especially for intensive checks like SSL.
- Future Work:
- Enriching checks with more detailed explanations of risks and actionable advice.
- Exploring the legal and ethical ramifications of providing information that could aid attackers.
- Expanding the set of implemented checks and supporting more browsers and mobile operating systems.
- Instructing the browser to click on internal links to gain a more comprehensive picture of each site.
Summary updated