MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents

Alexander Gurung, Spandana Gella, Alexandre Drouin, Issam H. Laradji, Perouz Taslakian, Rafael PardinasarXiv 2026 · arXiv

TL;DR

AI deep research agents that interleave private enterprise documents with public web searches pose a severe security risk: they inadvertently leak sensitive internal data through their public search queries. This paper introduces MosaicLeaks, a benchmark to quantify this "mosaic-effect" leakage, and proposes Privacy-Aware Deep Research (PA-DR), a reinforcement learning framework that slashes information leakage from 34.0% to 9.9% while maintaining high task accuracy.

What Problem Does This Paper Address?

When AI agents perform "deep research," they often need to synthesize information across both internal enterprise documents (like private financial reports) and external sources (like the public web). To do this, the agent generates web search queries based on what it learns internally.

The catch? These external queries are executed "in the open," meaning they can be monitored by adversaries, search engines, or network observers. While a single query might seem completely benign, an adversary can aggregate a stream of queries over time to reconstruct sensitive corporate secrets. This is known as the mosaic effect.

Prior work on agent privacy has largely relied on simple system prompting (e.g., telling the agent "don't leak secrets"), which performs poorly. Furthermore, standard reinforcement learning (RL) frameworks optimize purely for task accuracy, which actually incentivizes agents to become more leaky by forcing them to generate highly specific, revealing queries to find the right information.

Overview of how the mosaic effect leaks information across different axes.

Methods

The authors tackle this problem by framing privacy preservation as a joint optimization problem alongside task performance using Reinforcement Learning via Verifiable Rewards (RLVR). Their framework, Privacy-Aware Deep Research (PA-DR), introduces two key innovations:

  1. Situational Rewards: Instead of using a single sparse reward at the end of a long research task, they implement dense, stage-by-stage rewards for the agent's Plan (query generation) and Choose (document selection) steps. This provides a much cleaner gradient signal during training.
  2. Privacy Leakage Penalty: They train a lightweight binary classifier to act as a privacy reward model. This classifier estimates the probability that a batch of web queries leaks private information. Crucially, the reward accounts for both Direct Leakage (the current query batch is inherently leaky) and Mosaic Leakage (the current query batch becomes leaky when combined with queries from previous hops).

Datasets

The paper introduces MosaicLeaks, a new evaluation benchmark designed specifically to test multi-hop research dependencies between local and external data.

  • The Benchmark: It consists of 1,001 multi-hop research questions (comprising 3,403 individual hops). It chains private enterprise documents from DRBench with a public web corpus from BrowseComp-Plus.
  • Private QA Set: To evaluate leakage, the authors generated 3,686 ground-truth private question-answer pairs from the internal documents to define exactly what information needs protecting.
  • Classifier Training Data: They collected a dataset of 26,734 agent query sequences, judged by a powerful LLM adversary, to train their privacy reward model.

A comparison of MosaicLeaks against existing agent benchmarks highlights its unique focus on combining multi-hop, local-to-web tasks with mosaic privacy evaluations:

Comparison of MosaicLeaks with previous work.

Key Insights and Ideas

  • Optimizing for Performance Worsens Privacy: Standard RL training makes agents more accurate but significantly more dangerous. To get the right answer, the agent learns to embed exact metrics, dates, and codenames into public search bars.
  • Prompting is a Blunt Instrument: Zero-shot privacy prompts don't actually teach the model how to write safer queries; they simply scare the model into making fewer queries overall, which harms task performance.
  • Learning to Abstract: PA-DR successfully teaches the model a sophisticated defense mechanism. Instead of suppressing queries, the trained agent learns to abstract them—omitting specific dates or exact percentages while retaining enough general context to retrieve the correct public documents.

Key Results

The authors evaluated their method by training a Qwen3-4B-Instruct model and measuring both Strict Chain Success (solving the entire multi-hop task correctly) and Privacy Leakage (adversary successfully guessing private answers or facts).

  • The Baseline: The un-tuned base model achieved a 48.7% success rate but suffered a 34.0% privacy leakage rate.
  • Task-Only Training: Training purely for performance boosted success to 59.3% but caused privacy leakage to skyrocket to 51.7%.
  • PA-DR (The Fix): The privacy-aware RL framework pushed the Pareto frontier, achieving a 58.7% success rate while plummeting privacy leakage down to just 9.9% (and further down to 7.6% when paired with a privacy prompt).
The Pareto frontier showing how PA-DR achieves high success while minimizing privacy leakage.

Limitations and Future Work

While promising, the authors note several limitations:

  • Human Effort: Despite automated generation pipelines, significant human effort was still required to validate the quality and answerability of the benchmark chains.
  • Domain Constraints: The evaluation is restricted to the three company contexts available within DRBench.
  • Task Scope: The benchmark focuses strictly on multi-hop question answering rather than open-ended, long-form research report generation.
  • Harness Dependency: The training was restricted to the specific Plan and Choose stages of the authors' agent harness; future work could explore training the document reading and final resolution stages as well.

Summary updated

Abstract

Deep research agents increasingly combine private local documents with external tools like web retrieval, creating a privacy risk: an agent's external queries may leak sensitive information from its local context. This risk is amplified by the mosaic effect, where individual queries may appear harmless but become revealing in aggregate. We introduce MosaicLeaks, a benchmark of 1,001 multi-hop deep research tasks that chain private enterprise documents and a public web corpus, forcing agents to make external queries that depend on local information. We evaluate leakage with an adversary LLM that observes only the agent's external queries and attempts to infer private information at three levels: the agent's research intent, answers to specific private questions and verifiable claims about the enterprise documents. We find that models across families and sizes frequently leak at all three levels, that zero-shot privacy prompting reduces but does not eliminate leakage and that reinforcement learning for task performance alone worsens leakage. To address this, we propose Privacy-Aware Deep Research (PA-DR), an RL framework that combines situational rewards for task success with a learned privacy classifier to provide dense credit assignment over both per-query and mosaic-level leakage. Training Qwen3-4B-Instruct with PA-DR improves accuracy from 48.7% to 58.7% and reduces answer and full-information leakage from 34.0% to 9.9%.
BibTeX
@misc{gurung2026mosaicleaks,
  author = {Alexander Gurung and Spandana Gella and Alexandre Drouin and Issam H. Laradji and Perouz Taslakian and Rafael Pardinas},
  title = {{MosaicLeaks:Privacy Risks in Querying-in-the-Open for Deep Research Agents}},
  year = {2026},
  eprint = {2605.30727},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.30727},
}
Figure