RedAct: Redacting Agent Capability Traces for Procedural Skill Protection

Protecting the Secret Sauce of AI Agents

When AI agents share their work logs, they might accidentally leak secret formulas or specialized methods. This research introduces a way to "redact" those logs—blurring the secret details while keeping enough info to prove the work was done correctly—and adds a digital watermark to track if someone steals the methods.

As tool-using agents move from research prototypes to industrial deployments, they are being built around proprietary data and expert-refined workflows. These "agent skills"—structured packages of procedural knowledge like custom code templates or domain-specific heuristics—are the intellectual property of the companies that build them. However, to ensure accountability and help developers debug failures, these agents must produce execution traces: detailed logs of every reasoning step, tool invocation, and error-recovery attempt. This creates a fundamental tension. How do you release enough detail for a human to audit the agent's work without handing over the proprietary "secret sauce" that makes the agent effective?

The vulnerability of transparent logs

The central question investigated by the authors is whether public agent traces can be treated as security interfaces. Specifically, they ask how useful agent traces can be released without exposing reusable protected skills, while still preserving the evidence required for auditing.

Previously, the field viewed trace release through a binary lens. One could release "answer-only" logs, which hide everything except the final result, or "raw" traces, which provide full transparency. As shown in, raw traces are highly informative for debugging. However, they also serve as a blueprint for competitors. A downstream party with access to these raw traces can use "skill distillation"—a process of extracting reusable procedures from observations—to recover proprietary formulas, threshold configurations, and tool-call sequences. Essentially, the very detail that makes a trace useful for a developer makes it a goldmine for an adversary looking to clone the agent's capabilities.

Quantifying the leakage risk

To measure this risk, the authors constructed CAPTRACEBENCH. This is a specialized benchmark consisting of 75 long-horizon tasks across seven domains, such as life sciences, finance, and software security. This benchmark includes 154 curated skills that are difficult to recover from simple instructions alone.

The researchers conducted an investigation into how much "utility" is leaked when raw traces are made public. They tested several downstream reuse methods, including single-agent synthesis (creating new skill documents from traces) and trajectory fine-tuning (training a new model on the traces). The results were stark. The authors report that raw traces allow downstream methods to recover significant procedural value. Specifically, Normalized Skill Transfer (NST)—a metric measuring how much a student agent gains compared to a baseline with no skills—ranged from 44.7% to 67.1% .

This means a competitor could effectively "steal" a large portion of the agent's expertise just by reading its logs. This confirmed that traces leak operational knowledge, such as specific tool choices and intermediate validation routines, rather than just the final answers.

Introducing the REDACT framework

To address this, the authors propose REDACT (Redacting Agent Capability Traces), a two-layer defense mechanism. The first layer is "Key-Information-Guided Rewriting." Instead of simply deleting text, an LLM-based locator identifies sensitive items—like specific mathematical constants or proprietary API parameters. A rewriter then replaces them with abstract, natural-sounding descriptions. This is analogous to a redacted legal document that replaces a specific person's name with "[Defendant]" rather than leaving a blank space. The narrative flow remains intact, but the identity is protected.

The second layer is "Behavioral Watermarking." The authors embed functionally neutral behaviors—small, harmless actions or phrases—into the traces. If a competitor later trains a model using these traces, the watermark can be detected in the new model's behavior. The researchers found that these standalone watermarks achieved a true detection rate of 93.6% to 100.0%. Furthermore, the false alarm rate stayed at or below 1.9% [Table 3]. This indicates the system can accurately identify stolen traces without wrongly accusing legitimate users.

Neutralizing the skill transfer advantage

The effectiveness of REDACT is demonstrated by its ability to suppress the advantages gained from stolen traces. The authors report that after applying REDACT, the NST across all tested reuse methods dropped from the 44.7–67.1% range to below the "no-skill" baseline .

This means a student agent trained on protected traces performs no better than an agent that had no access to any specialized skills at all. The protection effectively breaks the link between the trace and the learned skill.

Crucially, the authors show that this protection does not destroy the trace's utility for humans. While REDACT successfully removes approximately 70.6% of sensitive key items, it preserves 91.0% to 96.6% of the "audit-critical" evidence, such as tool names and final answers .

This allows a developer to still verify that the agent followed the correct sequence of steps and reached a valid conclusion. The specific proprietary parameters used during those steps are obscured, but the "proof of work" remains.

Implications for agent security

If these results hold in more complex, noisy real-world environments, the implications for the industry are significant. First, it suggests that companies can participate in the open-source ecosystem. They can share traces for transparency and safety research without fear of immediate IP theft. Second, it establishes a new paradigm for provenance. Instead of relying on impossible-to-track model weights, companies can use behavioral signatures to identify unauthorized reuse of their data.

However, the paper does not explore how an adversary might attempt to "clean" the traces to remove watermarks. For example, an attacker might paraphrase the injected phrases. A logical follow-up experiment would be to test the robustness of the behavioral watermarks against aggressive linguistic transformations or multi-step fine-tuning aimed at stripping away non-essential behaviors.

Figures from the paper