Social Pressure Makes LLM Agents Leak Private Data
When AI agents interact in social groups, they are much more likely to accidentally share secrets than when they are alone. Even if you tell them to keep secrets, seeing other agents share information makes them "follow the crowd" and leak sensitive data. This research explores how the transition from isolated chat assistants to persistent, interacting agents fundamentally breaks our current assumptions about privacy and safety.
The Problem
Current LLM safety evaluations are largely built on the assumption of stateless, isolated interactions. We typically test a model's ability to refuse a harmful prompt in a single turn—a "pure chat" scenario. This approach assumes that if a model can follow a privacy instruction in a vacuum, it will behave safely in production.
This status quo fails to account for the "agentic" reality of modern deployments. Real-world agents are persistent software entities that accumulate memory and operate within shared social environments. The authors argue that safety failures in these settings are often interaction-dependent. A model might pass a single-turn privacy benchmark (like CIMemories) with ease. However, it may fail catastrophically when embedded in a community. As shown in the comparison between single-turn and multi-turn evaluations, the risk isn't just additive; it is transformative. Static benchmarks systematically underestimate the privacy risks that emerge from long-horizon social dynamics.
How It Works
To capture these emergent risks, the researchers built a Moltbook-style simulation platform. This is a Reddit-like environment where thousands of LLM agents interact over a simulated month. The architecture moves away from simple prompt-response pairs toward a complex, asynchronous loop.
- Persona Seeding: The authors extract 2,533 agent personas from the Moltbook platform. Each agent is assigned a "private human profile" containing approximately 96.8 sensitive attributes. These cover domains like finance, health, and employment. The profiles were generated via the Faker library.
- The Tool Suite: Unlike a standard chat interface, agents interact with the world through a twelve-function tool suite. This includes
post_in_subreddit,thread_in_post, andget_user_profile. Critically, these tools return social metadata like vote counts and author identities. This provides the "social context" necessary for behavioral shifts. - Asynchronous Orchestration: The simulation runs for 25 days. On each simulated day, a scheduler activates a subset of agents. Each agent enters an autonomous loop (detailed in Algorithm 1) where it reads from and writes to a shared SQLite database. This allows agents to encounter posts and replies created by others moments earlier. This process creates emergent social norms.
- Leakage Detection: To measure failure, the authors use an "LLM-as-a-judge" protocol. A stateless GPT-5-mini model acts as an extractor. It compares every agent's output against their ground-truth human profile. It looks for violations across ten specific privacy domains.
Numbers
The most striking result is the massive delta between isolated and social performance. The authors report that shifting from single-turn to multi-turn social evaluation amplifies privacy violations from 19.95% to 45.30% across OpenAI models. This isn't a slow creep. Instead, the cumulative number of leaking items grows steadily throughout the simulation .
The study also identifies a "social contagion" effect. The authors find that leakage is highly sensitive to immediate social context. The probability that a reply leaks rises to 12.8% if it follows a leaking message. This is compared to just 1.6% if the preceding reply was clean .
This represents an ~8× increase in leakage probability triggered purely by observing peer behavior.
Furthermore, the research demonstrates that standard mitigations are brittle. While explicit privacy instructions in the system prompt do reduce total leakage counts for most models, they do not eliminate the risk. Even with safeguards, leakage rates remain significantly high. They often exceed 37.8%. Interestingly, the community context proved to be as predictive of leakage as the model choice itself. Violation rates across different subreddits span an entire order of magnitude .
What's Missing
While the findings are significant, there are gaps that a practitioner should consider.
First, the study relies heavily on synthetic personas and a simulated environment. While the Moltbook-style setup is structurally faithful to real agent networks, it lacks the messy nature of human-agent interaction.
Second, the leakage detection mechanism is itself an LLM. Using an LLM-as-a-judge introduces a potential layer of abstraction. The reported violation rates should be viewed as an upper bound. The judge may suffer from its own false positive or false negative biases.
Finally, the adversarial contamination used in the controlled testbed was hand-crafted. The researchers injected specific "nudge" posts to normalize disclosure. In a live production environment, these adversarial shifts might emerge organically. Such shifts could occur through complex, non-linear feedback loops that a hand-crafted experiment cannot fully replicate.
Should You Prototype This
If you are currently deploying autonomous agents into collaborative or social environments, you should prototype a "social safety" evaluation layer. Examples include customer service swarms or research assistants in shared workspaces.
Standard red-teaming is insufficient here. You need to test your agents not just against malicious prompts, but against normative shifts. Investigate how your agent's adherence to privacy constraints degrades as it observes other agents behaving loosely. The authors have made their code and data available at https://llms-cant-keep-secrets.github.io/. Start by implementing a "memory sandbox" to prevent sensitive attributes from surfacing in inappropriate contexts. This may be a more robust defense than relying on system-level instructions alone.