Abstract
Predicting image privacy is crucial for protecting social media users, yet many are hesitant to upload raw images to third-party services. This paper explores an alternative: predicting privacy using only image tags. We propose a framework that leverages BERT models enhanced by Knowledge Distillation (KD) and semi-supervised learning. Our results show that a distilled BERT model ($BERT_{KD}$) achieves state-of-the-art F1-scores and superior calibration. Notably, the semi-supervised approach allows the model to reach full-dataset performance using only 20% of the available labeled data, addressing the high cost of manual annotation.
Introduction
The rapid growth of social media has created a "privacy paradox." While users express concern over sensitive content being public, they are often unwilling to use automated privacy-prediction tools that require uploading raw image pixels to external servers. Image tags (e.g., "bedroom," "family," "vacation") offer a privacy-preserving alternative for analysis. However, tags are often sparse and lack the semantic context necessary for traditional models like Bag-of-Tags (BoT) or basic CNNs to accurately discern privacy boundaries.
This research addresses these challenges by applying Bidirectional Encoder Representations from Transformers (BERT) to the task of tag-based privacy prediction. By treating image tags as a sequence of text, we can leverage BERT's deep contextual understanding. To further improve efficiency and robustness, we incorporate Knowledge Distillation and semi-supervised learning techniques.
Methodology
The core of our approach is a Knowledge Distillation framework designed to transfer insights from a complex "Teacher" model to a "Student" model.
Knowledge Distillation Framework
We first train a Teacher BERT model on the labeled portion of the dataset. A Student BERT model, sharing the same architecture, is then trained using a composite loss function $L_S$. This loss combines "hard" labels (the ground truth $y$) and "soft" labels ($P_T$) provided by the Teacher. The soft labels represent the probability distribution over classes, capturing the Teacher's uncertainty and providing a richer signal for the Student to learn the decision boundary.
Semi-Supervised Learning
To mitigate the need for extensive manual labeling, we employ the Teacher model to generate pseudo-labels for a large pool of unlabeled data. The Student model is then trained on this augmented dataset, allowing it to benefit from the underlying patterns in the unlabeled tags.
Experimental Setup
We evaluated our framework using the PicAlert dataset, which contains 32,000 Flickr images. These images were manually annotated by external viewers as either "private" (belonging to the personal sphere) or "public." The dataset was split into 22,000 training, 5,000 validation, and 5,000 testing samples. Pre-processing involved filtering rare tags and cleaning noise to ensure high-quality input for the BERT tokenizer.
Results and Discussion
Our experiments demonstrate significant improvements across several key metrics.
Performance and Data Efficiency
The distilled model, $BERT_{KD}$, achieved a private-class F1-score of 0.681, outperforming standard BERT (0.664) and baseline models like TagCNN (0.629). A critical finding is the efficiency of the semi-supervised approach. As shown in Figure 2, the semi-supervised BERT model reaches the performance levels of a fully supervised model using only 20% of the labeled training data (approximately 4,400 samples).
Calibration and Robustness
Beyond accuracy, $BERT_{KD}$ exhibited better calibration, reducing Expected Calibration Error (ECE) by up to 2.62%. This ensures that the model's confidence scores are more reliable indicators of actual correctness. Furthermore, the model proved highly robust against adversarial "Synonym-based Attacks," maintaining an F1-score of 0.962 when tags were replaced with similar words, whereas standard models showed significant degradation.
Semantic Insights
Our analysis revealed a strong link between tag emotions and privacy. Tags associated with Trust, Joy, and Disgust (e.g., "innocent," "messy") were predictive of private images, while tags evoking Fear and Surprise (e.g., "caution," "marvel") were more common in public content.
Conclusion
This work demonstrates that BERT, combined with Knowledge Distillation, is a powerful tool for privacy prediction based solely on image tags. Our framework not only improves accuracy but also significantly reduces the reliance on labeled data through semi-supervised learning. Future work will explore multi-modal architectures that combine tag-based analysis with selective image pixel processing to handle cases where tags are missing or ambiguous.
Summary updated