AI Delegates with a Dual Focus: Ensuring Privacy and Strategic Self-Disclosure

Xi Chen, Zhiyang Zhang, Fangkai Yang, Xiaoting Qin, Chao Du, Xi Cheng, Hangxin Liu, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang

Abstract

As LLM-based agents transition into autonomous social roles, they encounter a "Privacy Paradox"—the tension between safeguarding sensitive user data and the necessity of strategic self-disclosure for building social rapport. We present a novel multi-agent AI delegate system designed to navigate this balance. By isolating reasoning about disclosure from direct data access, our framework protects private information while effectively pursuing social goals. Evaluation against standard LLM baselines demonstrates superior privacy preservation and significantly improved alignment with human social preferences.

Introduction

The emergence of AI delegates capable of handling social interactions, such as networking or professional consultations, introduces complex privacy challenges. Unlike transactional tasks, social interaction relies on the "dance" of self-disclosure to establish trust. Existing privacy-preserving models typically adopt a defensive stance, minimizing disclosure at all costs. However, an overly reticent agent fails to build relationships, while an oversharing one risks exposing sensitive data like health status or financial details. This paper addresses the need for a nuanced framework that can handle persistent social pressure and adversarial inquiries without sacrificing social utility.

Methods

The proposed framework utilizes a multi-agent architecture to decouple the decision-making process from data retrieval. This architecture is composed of four specialized roles:

  • Situation Assessor: Analyzes conversation history and social context (e.g., stranger vs. friend) to identify active social goals.
  • Strategy Maker: Evaluates social norms and user-defined rules to determine the optimal disclosure strategy, including depth and timing.
  • Privacy Retriever: Acts as the system's security guard. It is the only module with access to the user's full private data, cross-referencing the strategy with "deny-list" rules before fetching specific information.
  • Responder: Synthesizes the retrieved data and strategy into a natural-sounding final response.
The multi-agent architecture of the AI Delegate system, highlighting the separation of strategy and data retrieval.

Evaluation and Results

The system was tested using a benchmark of 100 social scenarios, categorized into "passive" (responding to inquiries) and "proactive" (initiating disclosure) cases. The evaluation involved both LLM-as-a-judge (GPT-4o, Qwen2.5, DeepSeek-V3) and a human user study with 50 participants.

  • Superior Privacy Protection: In passive scenarios, the system achieved a privacy protection score of 4.52/5.0, significantly outperforming the GPT-4o baseline (4.20/5.0). Qualitative analysis showed the delegate maintained boundaries even when partners became persistent or rude.
  • Better Social Alignment: In proactive scenarios, the system's strategies were more human-like. Using KL divergence to measure alignment, the proposed model (0.64) was significantly more aligned with human preferences than the baseline (0.75).
Distribution of Privacy Protection and Context Alignment scores for both passive and proactive scenarios, comparing the proposed system against the baseline.

Discussion and Future Work

While the dual-focus approach effectively mitigates the risk of accidental leaks due to adversarial "jailbreaking," several limitations remain. The current evaluation relies on third-party observations rather than active participants, and the model is currently optimized for one-on-one interactions. Future work will explore multi-party group dynamics and long-term memory to manage evolving social relationships over time. This work demonstrates that decoupling reasoning from data access is a robust strategy for building trustworthy and socially capable AI delegates.

Summary updated

Abstract

Large language model (LLM)-based AI delegates are increasingly utilized to act on behalf of users, assisting them with a wide range of tasks through conversational interfaces. Despite their advantages, concerns arise regarding the potential risk of privacy leaks, particularly in scenarios involving social interactions. While existing research has focused on protecting privacy by limiting the access of AI delegates to sensitive user information, many social scenarios require disclosing private details to achieve desired social goals, necessitating a balance between privacy protection and disclosure. To address this challenge, we first conduct a pilot study to investigate user perceptions of AI delegates across various social relations and task scenarios, and then propose a novel AI delegate system that enables privacy-conscious self-disclosure. Our user study demonstrates that the proposed AI delegate strategically protects privacy, pioneering its use in diverse and dynamic social interactions.
BibTeX
@misc{chen2024ai,
  author = {Xi Chen and Zhiyang Zhang and Fangkai Yang and Xiaoting Qin and Chao Du and Xi Cheng and Hangxin Liu and Qingwei Lin and Saravan Rajmohan and Dongmei Zhang and Qi Zhang},
  title = {{AI Delegates with a Dual Focus: Ensuring Privacy and Strategic Self-Disclosure}},
  year = {2024},
  eprint = {2409.17642},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.17642},
}
Figure