Demystifying Hidden Privacy Settings in Mobile Apps

Yi Chen, Mingming Zha, Nan Zhang, Dandan Xu, Qianqian Zhao, Xuan Feng, Kan Yuan, Fnu Suya, Yuan Tian, Kai Chen, XiaoFeng Wang, Wei ZouIEEE Symposium on Security and Privacy 2019 · OpenReview

Abstract

Mobile apps include privacy settings that allow their users to configure how their data should be shared. These settings, however, are often hard to locate and hard to understand by the users, even in popular apps, such as Facebook. More seriously, they are often set to share user data by default, exposing her privacy without proper consent. In this paper, we report the first systematic study on the problem, which is made possible through an in-depth analysis of user perception of the privacy settings. More specifically, we first conduct two user studies (involving nearly one thousand users) to understand privacy settings from the user's perspective, and identify these hard-to-find settings. Then we select 14 features that uniquely characterize such hidden privacy settings and utilize a novel technique called semantics-based UI tracing to extract them from a given app. On top of these features, a classifier is trained to automatically discover the hidden privacy settings, which together with other innovations, has been implemented into a tool called Hound. Over our labeled data set, the tool achieves an accuracy of 93.54%. Further running it on 100,000 latest apps from both Google Play and third-party markets, we find that over a third (36.29%) of the privacy settings identified from these apps are “hidden”. Looking into these settings, we observe that they become hard to discover and hard to understand primarily due to the problematic categorization on the apps' user interfaces and/or confusing descriptions. Further importantly, though more privacy options have been offered to the user over time, also discovered is the persistence of their usability issue, which becomes even more serious, e.g., originally easy-to-find settings now harder to locate. And among all such hidden privacy settings, 82.16% are set to leak user privacy by default. We provide suggestions for improving the usability of these privacy settings at the end of our study.

Introduction

While mobile applications offer increasingly granular privacy controls, a significant "usability-privacy gap" persists. Users who wish to protect their data often find themselves thwarted by interfaces that bury critical settings under layers of menus or use ambiguous terminology. This paper presents a systematic investigation into these "hidden" privacy settings, quantifying their prevalence and identifying the design patterns that make them difficult to find.

Methodology: The Hound Framework

To automate the discovery and evaluation of privacy settings, we developed Hound, a tool that utilizes semantics-based UI tracing. Hound consists of two primary modules:

  1. Privacy Setting Discoverer (PSD): This module uses static analysis to extract UI widgets and their associated text, employing a Support Vector Machine (SVM) classifier to identify privacy-related controls.
  2. Hidden Privacy Setting Identifier (HPSI): This core component reconstructs the "UI-path" a user must navigate to reach a setting. By matching the text on a button (an indicator) to the title of the subsequent screen, Hound can map the logical flow of an app's interface.
Semantics-based UI tracing: Hound reconstructs the path from the home screen to a privacy setting by matching UI indicators to screen titles.

Characterizing "Hiddenness"

Based on user studies involving nearly 1,000 participants, we identified 14 key features that characterize hidden settings. These include UI-path length, visual clutter (the number of elements on a page), and indicator clarity. Our analysis found that non-standard icons—such as using a gift box or a "Like" thumb to represent settings—significantly increase the difficulty for users.

Standard icons for settings (gear wheels) and profiles (portraits) used to evaluate indicator clarity.

Large-Scale Findings

We deployed Hound on a dataset of 100,000 apps from Google Play and third-party markets. The results reveal a pervasive issue:

  • Prevalence: 36.29% of all identified privacy settings were classified as hidden.
  • Default Leakage: Most alarmingly, 82.16% of these hidden settings were configured to leak user data by default. Because they are difficult to locate, users are unlikely to ever opt-out.
  • Problematic Categorization: Hiddenness is often a result of developers categorizing settings by function (e.g., placing message privacy under "Message Settings") rather than user expectation (expecting all privacy controls in a central "Privacy" menu).
  • Evolutionary Trends: A longitudinal study of 5,485 app pairs showed that privacy settings often become harder to find over time as apps grow in complexity, with previously accessible settings being moved to deeper sub-menus.

Conclusion

The prevalence of hidden privacy settings represents a significant challenge to user agency. Our study demonstrates that "privacy by design" is frequently undermined by "leakage by default" and poor UI categorization. We recommend that developers centralize privacy controls and use standardized indicators to bridge the usability-privacy gap.

Summary updated

Abstract

Mobile apps include privacy settings that allow their users to configure how their data should be shared. These settings, however, are often hard to locate and hard to understand by the users, even in popular apps, such as Facebook. More seriously, they are often set to share user data by default, exposing her privacy without proper consent. In this paper, we report the first systematic study on the problem, which is made possible through an in-depth analysis of user perception of the privacy settings. More specifically, we first conduct two user studies (involving nearly one thousand users) to understand privacy settings from the user's perspective, and identify these hard-to-find settings. Then we select 14 features that uniquely characterize such hidden privacy settings and utilize a novel technique called semantics- based UI tracing to extract them from a given app. On top of these features, a classifier is trained to automatically discover the hidden privacy settings, which together with other innovations, has been implemented into a tool called Hound. Over our labeled data set, the tool achieves an accuracy of 93.54%. Further running it on 100,000 latest apps from both Google Play and third-party markets, we find that over a third (36.29%) of the privacy settings identified from these apps are “hidden”. Looking into these settings, we observe that they become hard to discover and hard to understand primarily due to the problematic categorization on the apps' user interfaces and/or confusing descriptions. Further importantly, though more privacy options have been offered to the user over time, also discovered is the persistence of their usability issue, which becomes even more serious, e.g., originally easy-to-find settings now harder to locate. And among all such hidden privacy settings, 82.16% are set to leak user privacy by default. We provide suggestions for improving the usability of these privacy settings at the end of our study.
BibTeX
@inproceedings{chen2019demystifying,
  author = {Yi Chen and Mingming Zha and Nan Zhang and Dandan Xu and Qianqian Zhao and Xuan Feng and Kan Yuan and Fnu Suya and Yuan Tian and Kai Chen and XiaoFeng Wang and Wei Zou},
  title = {{Demystifying Hidden Privacy Settings in Mobile Apps}},
  booktitle = {IEEE Symposium on Security and Privacy},
  year = {2019},
  url = {https://openreview.net/forum?id=0u5EjtK8ESI},
}

Loading related content...

Figure