Your AI research agent is leaking private data through every web query it makes

Deep research agents, AI systems that combine private local documents with external web retrieval to answer complex queries, are increasingly deployed in enterprise settings. But a new study, MosaicLeaks, reveals that these agents pose a significant privacy risk: an adversary observing only the agent's outbound web queries can reconstruct sensitive internal information through the classic mosaic effect.

The Mosaic Effect in AI Research

Consider a healthcare research agent working on a routine question. As it works, it fires off a handful of seemingly innocuous web searches, one references a cloud migration milestone, another a January 2024 security disclosure, a third narrows down a vendor. No single query reveals the secret. But anyone watching the agent's outbound traffic can reassemble the fragments: MediConn had migrated 70% of its infrastructure to the cloud by January 2025, a fact that existed only in private documents.

This is the central failure mode at the heart of MosaicLeaks, a benchmark and training method developed by researchers from various institutions. The study treats web queries as the leakage channel: the adversary never sees the private documents or the agent's reasoning, only the cumulative query log, and attempts to infer private enterprise information.

Measuring Three Levels of Leakage

The benchmark categorizes leakage into three distinct types:

• Intent leakage: The adversary can infer the private research questions or goals the agent was pursuing.
• Answer leakage: The adversary can answer specific private questions just from the query log, without seeing the original documents.
• Full-information leakage: The strongest case, an observer can discover and state verifiable private facts without being told what to look for.

These represent an escalating level of concern. Intent leakage reveals what the agent is investigating. Answer leakage means the query log holds enough to answer a private question someone already has in hand. Full-information leakage is the most dangerous: the observer can discover private facts without any prior knowledge.

Prompting Privacy Fails

One might assume an easy fix: just instruct the agent not to leak. The researchers tested exactly that, adding a line to the Plan prompt warning the agent against issuing web queries that leak local information. The results were disappointing.

For Qwen3-4B, the prompt lowered answer/full-information leakage from 34.0% to 25.5%, but strict chain success, the share of chains where every hop is answered correctly, dropped from 48.7% to 44.5%. The primary behavioral change was simply fewer web queries, not consistently safer query construction. The prompt helped slightly for some models, but significant leakage remained.

The Performance-Privacy Paradox

When the researchers trained the agent purely to improve task performance, strict chain success rose from 48.7% to 59.3%. But answer/full-information leakage climbed alongside it, from 34.0% to 51.7%. The model had learned to pack more context into its web queries, richer queries helped retrieve the right document but gave the adversary more fragments to reassemble.

"This is the central tension MosaicLeaks exposes. A more informative query is often better for the task and worse for privacy."

PA-DR: Training Privacy Into the Agent

The authors propose a new method called Privacy-Aware Deep Research (PA-DR), which combines two rewards. The first is a situational task reward that judges each model call against other calls made at the same stage and hop, with the same information available. This provides much more precise credit assignment compared to traditional outcome-based rewards.

The second is a learned privacy reward. Whenever the agent produces web queries, a Qwen3-4B classifier estimates two risks: whether the current queries leak private information directly, and whether adding them to the existing query log creates a new mosaic leak. PA-DR penalizes the larger of the two risks.

The results are striking:

Method	Strict Chain Success	Answer or Full-Information Leakage
Base Qwen3-4B	48.7%	34.0%
Task Reward Only	59.3%	51.7%
Task + PA-DR Reward	58.7%	9.9%

That 9.9% is lower than the untrained base model's own 34.0%. Training for privacy did not simply cancel the leakage that training for performance introduced, it left the agent leaking less than it did at the start.

Notably, PA-DR achieves this without searching less. The agent issues more web queries than the base model, but those queries drop revealing details like specific metrics ("15%" or "2024") and clues about the kind of answer it is looking for. It still finds the right public documents, it just stops carrying private fragments along in the query text.

Sample Efficiency Gains

The situational task reward also pays dividends during training. Because it compares matching calls instead of scoring a whole rollout once, it assigns credit far more precisely without needing a separate value model. The situational task reward reaches the same task performance as outcome-only RL with roughly 5-6x fewer generated training samples, and PA-DR keeps that efficiency while adding the privacy gain.

Limitations and Takeaways

MosaicLeaks is a controlled benchmark, not a measurement of leakage in deployed systems. The enterprise documents are synthetic, the web corpus is fixed, the chains span three company contexts, and every result comes from a single agent harness running multi-hop question answering rather than open-ended research. That control is what makes leakage measurable hop by hop, but broader tasks, real deployments, and other agent designs still need their own study.

The study's core takeaway is clear: you can't prompt privacy in, you have to train it in. Telling an agent to be careful barely moves the needle, while rewarding how it constructs each query cuts leakage by more than 3x and leaves task success essentially intact.

The mosaic effect comes from how an agent searches over time, and this research demonstrates that it is something you can measure, assign credit to, and train down.