SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing

Beyond Fixed Risk Categories: The Challenge of Custom AI Safety

Current AI safety tools usually follow a fixed set of rules. However, real companies need tools that can follow custom, changing policies. SafePyramid is a new test. It checks if AI can understand and apply complex, custom rules. It even tests if they work when rules are written in entirely new or fictional ways.

Most Large Language Model (LLM) deployments rely on "guardrails." These are software layers that intercept user inputs and model outputs to flag unsafe content. Traditionally, these guardrails operate on fixed taxonomies. These are static checklists of risks like "hate speech" or "sexual content." While effective for general safety, this approach is rigid. A niche application often requires highly specific, nuanced policies. These policies do not fit into broad, predefined categories.

The industry is shifting toward "in-context policy guardrailing." In this setting, safety rules are provided as part of the prompt at inference time (the moment the model generates a response). This allows an organization to update its safety standards instantly. They can simply edit a text document instead of retraining a model. However, as this research demonstrates, executing these custom instructions is much harder than it looks.

The rigidity of fixed taxonomies

Standard guardrails function like a basic airport security scanner. They are programmed to look for a specific, unchanging list of prohibited items. If a new type of contraband emerges, the scanner is useless until it is upgraded. In the context of LLMs, current guardrails map interactions to coarse risk categories .

They might label something as "Privacy" or "Fraud." They do not adhere to the granular, auditable rules a specific company might demand.

The gap lies in the transition from classification to execution. Most existing benchmarks, such as DynaBench, treat guardrailing as a task of following general instructions. Examples include "don't use emojis." This is fundamentally different from enforcing a legalistic safety policy. There is a functional difference between a model that knows what "spam" is and a model that can navigate a 30-rule document. Such a document might detail exactly which types of user metadata are permissible under a specific corporate mandate.

A hierarchical stress test for rules

To bridge this gap, the authors of SafePyramid developed a benchmark. It does not just ask if a model is "safe." It asks how precisely it follows a provided rulebook. The researchers organized the benchmark into a three-level hierarchy .

This design isolates specific cognitive failures.

1. Level 0 (L0): Individual Rule Understanding. This level tests the basics. The model sees a rule and a conversation. It must decide if the rule was violated. To prevent the model from "guessing" based on topic, the authors include "distractor rules" .

These are rules that are topically related but are not actually triggered. 2. Level 1 (L1): Resolving Rule Dependencies. Real policies are rarely flat lists. They are webs of logic. L1 introduces "exception rules," which can waive a violation. It also includes "conditional rules," which make a rule stricter if certain context is met . This forces the model to perform logical reasoning. 3. Level 2 (L2): Adapting to Novel Frameworks. To ensure models do not rely on pre-trained knowledge, the authors rewrite policies using fictional concepts. This forces the guardrail to actually read and learn the new framework provided in the context.

The benchmark is massive. It contains 1,000 multi-turn conversations across 10 domains. It includes 3,000 policies containing over 61,000 distinct rules.

Performance collapses under complexity

The results reveal a steep decline in capability as logic deepens. Even the strongest model, GPT-5.5, struggles to maintain precision. On Level 0, GPT-5.5 achieved an exact-match Rule Matching Rate (RMR) of 54.0%. This means it correctly identified the full set of violated rules in only about half of the cases. This plummeted to 35.3% at Level 1. It fell to a mere 12.9% at Level 2 [Table 3].

This monotonic drop suggests a major bottleneck. The issue is not just understanding a rule. It is managing how rules interact. The authors find that errors at the highest level are common. Models often treat conditional rules as violations themselves. Instead, they should use them as logic gates to evaluate base rules.

The study also explores the cost-performance trade-off. Frontier models like GPT-5.5 offer the highest accuracy. However, they come with a massive price tag. Some cheaper models, such as Gemini-3.5-Flash, provide a more efficient path .

They achieve near-50% RMR at the lowest cost. Furthermore, "per-rule evaluation" can boost performance. This method asks a model to check one rule at a time. This can significantly help smaller, specialized guard models [Table 4].

Limitations of the pyramid

While SafePyramid is rigorous, the authors acknowledge several boundaries. First, the benchmark is currently limited to text-based interactions. Real-world guardrails might need to handle multimodal inputs (inputs like images or audio). This study does not explore that capability.

Second, the study lacks a human baseline. The researchers did not measure how well trained human policy auditors would perform. Without knowing the "human ceiling," it is difficult to measure the exact gap. Finally, the 10 domains represent only a snapshot. They cannot cover the infinite variety of possible corporate or legal policies.

The verdict: Not ready for prime time

If you want to deploy a fully autonomous, in-context policy guardrail today, the answer is: not yet.

The SafePyramid study proves that even advanced frontier models fail. They cannot reliably execute complex or novel policy frameworks. Current models excel at recognizing "bad things" from their training data. However, they are brittle when following a brand-new, logically complex rulebook.

For practitioners, the takeaway is to favor hybrid approaches. The authors suggest using "agentic harnesses." These are tools that allow a model to decompose a policy. They enable the model to verify rules step-by-step [Table 5]. Until models master rule dependencies and novel abstractions, guardrailing will remain difficult. It will likely require human-in-the-loop systems or highly specialized models.

Code and datasets for this research are reportedly available at https://github.com/bytedance/safepyramid.

Figures from the paper