PolicyGuard: A Dialogue-Grounded Sub-Agent Verifier for Policy Adherence in LLM Agents

Beyond Safety Filters: Teaching AI Agents to Follow the Rules

When AI agents handle real-world tasks like booking flights, they don't just need to be "safe." They need to be compliant. Much of the industry focuses on preventing toxic content. However, a different failure occurs when an agent skips company procedure. Examples include failing to ask for a user's ID or skipping a mandatory insurance offer.

Currently, developers use "guards" to inspect individual tool calls (commands sent to external software). These guards check if the arguments look correct. However, these guards are often blind to the actual conversation. If a policy requires a user to say "yes" before a booking, a guard looking only at the command will miss it. It cannot see if that verbal agreement actually happened.

A new study introduces POLICYGUARD, a specialized sub-agent verifier. It is designed to bridge this gap. Instead of just checking data in a single command, POLICYGUARD reads the entire conversation history. This ensures every procedural step is satisfied. The researchers report that this approach improves reliability. It increases successful task completion by up to 12.0 percentage points across several leading AI vendors.

The Blind Spot in Argument-Level Guarding

Most LLM safeguards catch adversarial attacks or toxic outputs. As the authors argue, company policy adherence is a different problem. In a typical business workflow, the user is not being malicious. The agent is not trying to be harmful. Instead, the failure is procedural. The agent might skip a prerequisite read or fail to obtain explicit consent.

As illustrated in, traditional "action gatekeepers" focus on the tool call itself.

They act like a security guard checking a passport. They can see if a name is valid. However, they cannot tell if the person agreed to terms ten minutes ago. This makes them unable to verify "process-level" requirements.

The authors find that about two-thirds of the requirements in the tau2-bench airline benchmark are process-level. These include tasks like "user explicitly confirmed the booking." They also include "identity verified via a read-only call" (a command that retrieves data without changing it). Because existing guards lack access to the dialogue, they are blind to the context required to enforce these rules.

A Verifier That Reads the Room

To address this, the authors propose POLICYGUARD. This is a third actor placed in the middle of the tool-calling loop. Rather than the agent communicating directly with the environment, the call is intercepted by the Verifier sub-agent.

The mechanism operates in two phases. First, during an offline stage, an LLM pipeline transforms a raw policy document into a structured YAML checklist .

This checklist breaks down the policy into specific requirements for each tool. For example, it might specify that an agent "must offer insurance."

Second, during the online execution phase, the Verifier performs three roles : 1. Conversation Awareness: It reads the full history of the agent-user dialogue. This ensures it sees the same context the agent sees. 2. Self-Reasoning: It evaluates the pending tool call against both the raw policy text and the synthesized checklist. 3. Behavior-Driving Remediation: This is a vital feature. Instead of a generic "Access Denied" error, the Verifier returns a specific message. If an agent tries to book a flight without offering insurance, the Verifier tells it: "Offer insurance, then ask the user to confirm the full booking summary before booking."

This feedback loop allows the agent to fix its mistake in the very next turn.

Reliability Gains and Efficiency Trade-offs

The performance of POLICYGUARD was tested across three model families: GPT-5.4, Claude Sonnet 4.6, and Gemini 2.5 Pro. The researchers used the PASS4 metric. This measures the fraction of tasks where the agent succeeds consistently across four separate trials. High PASS4 scores mean the agent is highly reliable and predictable.

The results in [Table 2] show that the PG-CHECKLIST configuration consistently outperforms the baseline. Specifically, the authors report improvements of +12.0 pp for GPT-5.4, +6.0 pp for Sonnet 4.6, and +12.0 pp for Gemini 2.5 Pro. Notably, the Verifier achieved perfect recall on policy-violating tasks for GPT-5.4 and Sonnet 4.6 [Table 2]. This means it caught every instance where the agent tried to break a rule.

The paper also highlights an efficiency insight. Adding a Verifier increases the total number of LLM calls. However, the Verifier can be a "smaller, cheaper tier" than the main agent. In a small-scale ablation [Table 5], the authors found that using a "mini" version of the model still provided significant reliability lifts. This suggests that judging a single tool call is structurally lighter than managing a complex conversation.

Contextual Limits and Vulnerabilities

The authors note several limitations. First, the evaluation was confined to the tau2-bench airline domain. While this domain has many procedural rules, effectiveness might vary in other sectors. In a retail domain, the overhead of a Verifier might outweigh its benefits [Table 20].

Second, POLICYGUARD is a probabilistic system. It relies on an LLM to act as a judge. Therefore, its ability to catch violations is empirical rather than mathematically guaranteed. For mission-critical industries, a purely LLM-based verifier might not meet formal safety standards.

Finally, the system remains vulnerable to "indirect prompt injection." This happens when an attacker hides instructions within data returned by a tool. The authors demonstrate in that POLICYGUARD is reasonably robust against users lying about authority. However, it struggles when "poisoned" information is presented as a factual system note within a tool's output.

The Verdict: A Necessary Layer for Agentic Workflows

Is POLICYGUARD ready for production? For developers building customer-facing agents, the answer is likely yes. This is true provided the cost of additional LLM calls is accounted for.

The research shows that "guarding" an agent is not just about blocking bad inputs. It is about providing the agent with the conversational intelligence to stay on track. By moving from simple argument-checking to dialogue-grounded reasoning, POLICYGUARD offers a path toward making autonomous agents truly reliable. It shifts the paradigm from a blunt instrument to a sophisticated supervisor.

Figures from the paper