Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

Instead of forcing an AI to remember everything it sees during a search, researchers built a "harness" that acts like a digital notepad. This notepad handles the boring organization—managing candidate pools, tracking evidence, and deduplicating results. This allows the model to focus purely on making smart decisions about what to look for next.

In the field of agentic search, we typically train models as policies (mathematical functions mapping states to actions) that act over a growing transcript of text. The model must decide how to search. Simultaneously, it must perform the mental heavy lifting of remembering which documents are useful, which constraints remain unmet, and which claims have been verified. This formulation forces reinforcement learning (RL) to optimize both high-level semantic search decisions and routine, recoverable bookkeeping.

The authors of Harness-1 argue that this dual burden makes learning poorly conditioned and inefficient. They propose "stateful cognitive offloading." This involves moving the housekeeping into the environment itself. By separating the "brain" (the policy) from the "notepad" (the harness), they claim they can produce more capable and generalizable search agents.

The Problem

The status quo in retrieval-agent training treats the agent's interaction with tools as an append-only stream of text. As a search episode progresses, the transcript grows. This consumes the model's context window (the amount of text a model can process at once). It also forces the policy to reconstruct the search state from raw, noisy observations.

This approach breaks down in two specific ways. First, the reward signal becomes incredibly sparse and noisy. If a search fails, the RL algorithm struggles to discern the cause. It cannot easily tell if the failure stemmed from a poor search query, forgotten evidence, or a failure to verify a claim. Second, the policy often collapses into suboptimal behaviors. Without an explicit way to manage information, models frequently fall into "search-only" modes. They repeatedly issue queries because they lack a reliable mechanism to curate and organize the evidence they have already found.

How It Works

The core architectural shift in Harness-1 is the introduction of a stateful search harness. This harness maintains a persistent WORKINGMEMORY. Instead of the policy seeing a massive, undifferentiated transcript, it observes a structured, rendered view of the current search state.

As illustrated in, the system operates through a clear division of labor:

1. State Maintenance: The harness manages several internal data structures. These include a candidate pool ($P_t$), a curated output set ($C_t$) with importance tags, an evidence graph ($G_t$) summarizing entity relationships, and a verification cache ($V_t$).
2. Semantic Policy Actions: The 20B policy (built on gpt-oss-20b) makes high-level decisions. It doesn't just "search." It uses structured actions to curate (adding, removing, or tagging documents by importance), verify (writing claims to be checked against the text), and review (re-examining previously seen documents).
3. Derived-State Rendering: To keep the context manageable, the harness performs "cognitive offloading." It uses Sentence-BM25 compression (selecting the most relevant sentences based on keyword overlap) to boil down search results. It also employs a regex-based extractor (using pattern matching) to build a compact evidence graph. This graph highlights "bridge" documents. These are documents connecting multiple entities. This makes connections visible to the policy without a full-text reread.
4. Capacity Management: The harness enforces a hard cap of 30 documents for the curated set. When the limit is reached, the harness handles the mechanical logic. It evicts the lowest-importance documents based on the tags assigned by the policy.

Numbers

The authors report that Harness-1 (20B) achieves an average curated recall of 0.730 across eight benchmarks. This represents an +11.4 point improvement over the next strongest open-source search subagent .

This margin is significant for engineers looking for high-performing open-weights models. The model also remains competitive with much larger frontier-model searchers. It outperforms models like GPT-5.4 and Sonnet-4.6 in average curated recall under this protocol.

The most striking result is the model's ability to generalize. The paper finds that Harness-1's gains are significantly higher on held-out transfer benchmarks. These are tasks not seen during SFT (supervised fine-tuning) or RL. Specifically, the authors report a mean gain of +17.0 points on transfer tasks. This is much higher than the +7.9 points seen on source-family tasks .

This suggests the agent learns to manipulate an explicit search state. It is not just memorizing domain-specific patterns.

The training efficiency is also notable. Despite using relatively small amounts of data, the agent achieves high performance. It uses roughly 900 filtered SFT trajectories and 3,500 RL queries .

The harness provides a stable, structured interface. This allows the RL signal to act more effectively.

What's Missing

While the results are impressive, there are several gaps for practitioners:

• Brittle Extraction Logic: The evidence graph relies on a lightweight regex-based extractor for entities and dates. In highly technical domains, this could create blind spots. The harness might fail to surface the connections the policy needs.
• Proxy Verifier Reliability: The verify action uses an LLM-based proxy to perform entailment checks (determining if one text logically follows another). The authors admit this may struggle with highly technical or ambiguous claims. If the "notepad" provides incorrect verification results, the policy will learn to rely on faulty evidence.
• Missing Latency and Cost Data: The paper focuses on recall and accuracy. It does not report inference-time latency or throughput. It also does not specify the hardware costs of maintaining the stateful harness in production. For real-time RAG (Retrieval-Augmented Generation) systems, these costs matter.

Should You Prototype This

Yes, but with caveats. If you are building a high-precision search agent for specialized domains like finance or law, the principle is sound. Moving bookkeeping out of the context window saves optimization budget.

However, do not expect results by simply wrapping an LLM in a Python dictionary. The success of Harness-1 relies on tight integration. The policy's structured actions must work with the harness's algorithmic enforcement. If you prototype this, focus on the interface. Design a harness that provides a compact, actionable summary of the search state. Also, ensure your reward function explicitly incentivizes tool diversity. This prevents the policy from collapsing into a simple, repetitive search loop.

Code is reportedly available at https://github.com/pat-jj/harness-1.

Figures from the paper