Uncovering Scene Context for Predicting Privacy of Online Shared Images

Ashwini Tonge, Cornelia Caragea, Anna Cinzia SquicciariniAAAI 2018 · DOI

Introduction

As social media users share millions of images daily, the risk of inadvertently leaking sensitive information—such as home locations or private habits—continues to grow. Automated privacy prediction systems aim to mitigate this risk by warning users before they post sensitive content. However, traditional methods often struggle with the "semantic gap" between raw image data and privacy implications.

The Role of Scene Context

Existing approaches to privacy prediction primarily rely on user-provided tags or object detection. These methods face two major hurdles:

  1. Unreliable Metadata: User tags are frequently noisy, sparse, or entirely absent.
  2. Contextual Ambiguity: Objects themselves are often privacy-neutral. For instance, a "bed" in a furniture showroom is likely public, whereas a "bed" in a personal bedroom is inherently private.
Figure 2: A private scene where the context of a bedroom clarifies the privacy status of the objects within it.

The authors of this study argue that an effective AI must understand not just what is in an image (objects), but also where the image was taken (the scene).

Methodology

The researchers proposed a multi-stream approach using Convolutional Neural Networks (CNNs) to extract semantic features:

  • Object-centric Stream: An AlexNet model pre-trained on ImageNet (1,000 categories) identifies specific items.
  • Scene-centric Stream: An AlexNet model pre-trained on the Places2 dataset (365 categories) identifies the environment.
  • Feature Fusion: The top $k$ (where $k=2$ or $10$) most probable tags from both streams are combined with existing User Tags (UT).
  • Classification: These combined features are processed by a Support Vector Machine (SVM) with a Radial Basis Function (RBF) kernel to classify images as "public" or "private."

The study utilized the PicAlert dataset, consisting of 32,000 Flickr images manually labeled by external viewers, maintaining a realistic 3:1 public-to-private ratio.

Key Results

The primary insight of the research is that scene context acts as a critical "disambiguator." For example, the object tag "person" is neutral, but when paired with the scene tag "shower," the privacy risk becomes evident.

Figure 3: Detailed performance metrics comparing User Tags (UT), Scene Tags (ST), and Object Tags (OT) across different configurations.

The results demonstrate that scene and object tags provide complementary information. While object tags (OT) generally offer a higher boost than scene tags (ST) when added to user tags, the combination of all three (UT+OT+ST) yielded the highest performance. At $k=10$, the "All-in" model achieved an F1-score of 0.841, correctly identifying 854 more instances than the baseline model using only user tags.

Conclusion

By integrating scene context, the researchers bridged a significant gap in automated privacy prediction. Automated tagging serves as a vital "gap filler" for missing user metadata, providing a consistent baseline for classification. Future work may involve more sophisticated multimodal architectures and more granular privacy classifications beyond the binary public/private distinction.

Summary updated

BibTeX
@inproceedings{tonge2018uncovering,
  author = {Ashwini Tonge and Cornelia Caragea and Anna Cinzia Squicciarini},
  title = {{Uncovering Scene Context for Predicting Privacy of Online Shared Images}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.12180},
}

Loading related content...

Figure