Abstract
Automated profiling of social media users often relies on Multiple Instance Learning (MIL) to infer sensitive attributes from a collection of posts. While traditional privacy defenses focus on modifying individual items, this paper explores a "bag-level" defense: adding strategically chosen "pivoting" items to an existing profile. We evaluate three types of additions—adversarial noise, perturbed items, and natural content—across visual and audio datasets. Results show that adding a small number of items can significantly degrade profiling accuracy, even in black-box settings, offering a feasible path to privacy in an environment where data deletion is often impossible.
Introduction: The Shift to Bag-Level Privacy
Most research in adversarial machine learning for privacy focuses on the item-level, such as modifying a single image to prevent facial recognition. However, real-world profilers typically scrape entire user profiles. By using Multiple Instance Learning (MIL), these attackers aggregate information across a "bag" of items to infer high-level, sensitive attributes—such as the "Big Five" personality traits or emotional states—that are not readily apparent from a single post.
A significant hurdle for users is that once content is online, it is nearly impossible to delete completely due to caching and third-party scraping. This research addresses a critical question: Can we protect a user's privacy by adding new content to an existing profile to "pivot" the classifier’s prediction toward an incorrect result?
Proposed Defense: Pivoting by Addition
The authors propose shifting the focus from modifying existing data to adding new "pivoting" items. This approach is more reliable because users have full control over new uploads, whereas they may have lost control over historical data.
The study evaluates three types of pivoting additions with varying levels of stealth:
- Adversarial Noise (AdvN): Items consisting entirely of optimized noise. These are highly effective but easily detectable by humans.
- Adversarially Perturbed Items (AdvPI): Real images or audio clips modified with slight, human-imperceptible perturbations optimized to affect the entire bag's classification.
- Natural Items (NatI): Unmodified, "clean" items selected from a background pool. The system identifies items that naturally pull the classifier's prediction in a different direction. This is the most stealthy approach, as the added content appears perfectly normal to observers.
Experimental Setup
The defense was tested against four threat models, ranging from White-box (full access to the attacker's model) to Black-box (no knowledge of the attacker's architecture). The researchers utilized three primary datasets:
- Personality Profile (PsychoFlickr): Images annotated with "Big Five" personality traits.
- Emotion Profile: Profiles categorized by dominant emotions (e.g., anger, joy).
- Speech Gender Profile (LibriSpeech): Demonstrating the technique's applicability to audio data.
Key Insights and Results
The study yielded several critical findings regarding the nature of bag-level profiling and the effectiveness of the proposed defenses:
- The Power of Fusion: Modern MIL models using "early-fusion" (Deep Sets) or "intermediate-fusion" (Set Transformers) are significantly more effective at profiling than older "late-fusion" methods. This underscores the urgency of developing robust defenses.
- Effectiveness and Transferability: In white-box settings, AdvN and AdvPI reduced profiler accuracy from ~90% to near 0%. Crucially, in "near-black-box" scenarios, the methods remained effective. For the Emotion Profile, adding natural items (NatI) dropped attacker accuracy from 94% to 74%.
- Attention Shifting: Using interpretable models (AttMIL), the researchers found that pivoting additions work by "stealing" the model's attention. The classifier focuses on the deceptive new items rather than the original private content.
- Human Stealth: A user study indicated that in 50% of cases, human observers could not distinguish a "pivoted" profile from a normal one when Natural Items were used.
Conclusion and Future Work
This work demonstrates that "defense by addition" is a viable strategy for preserving privacy against automated profilers. While effective, limitations remain. A proactive attacker might attempt to filter out newer items, though identifying specific "pivots" is non-trivial. Future research may explore using Generative AI (e.g., DALL-E) to create custom pivoting items that match a user's specific aesthetic style while maintaining privacy.
Summary updated