HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry

Beyond Scaling: Evolving the Scaffolding of AI Agents

Modern AI agent performance depends heavily on more than just the raw intelligence of the underlying model. Instead, it relies on the "harness"—the collection of prompts, tools, memory, and control flows that mediate how a model observes, reasons, and acts. Currently, these harnesses are largely hand-crafted and static. When a new model or task arises, engineers must manually build bespoke scaffolding. Furthermore, the rich data generated during execution is rarely used to systematically improve the system.

The researchers behind HarnessX propose a shift in this paradigm. Rather than focusing solely on making the "brain" (the model) larger or smarter, they have developed a system to automatically evolve the "body" and "tools" (the harness) that surround it. By treating the runtime interface as a programmable, evolving object, HarnessX allows agents to learn from their own execution traces. This helps the system build better interfaces through direct experience.

The Anatomy of an Agent's Interface

To understand the breakthrough, one must first distinguish between the model and the harness. If the model is the cognitive core, providing reasoning and planning, the harness is the executive apparatus. It determines what the model perceives and which external tools it can invoke. It also manages how memory carries across long-running tasks.

In current engineering workflows, these components are often "architecturally entangled." This means that changing a single prompt template or tool wrapper can silently break the entire control loop. Furthermore, there is a disconnect between harness engineering and model training. The data collected while trying to optimize a harness is typically discarded. It is not fed back into the model to refine its capabilities. HarnessX aims to unify these two processes. It treats the harness as a first-class, composable object that can be assembled from modular, typed primitives.

The Mechanics of Symbolic Evolution

The core of the HarnessX framework is AEGIS, a multi-agent evolution engine. It transforms execution traces into structural updates. The authors frame this through an "operational mirror." This is a theoretical mapping that treats harness adaptation as a Reinforcement Learning (RL) problem played out in a "symbolic space." Unlike the model's parameter space, which consists of billions of continuous numbers, this symbolic space consists of discrete, human-readable artifacts like code, prompts, and tool definitions.

In this mapping, harness configurations act as "states." Typed code edits serve as "actions." The resulting execution traces and verifier scores provide the "feedback." As illustrated in, AEGIS operates through a four-stage pipeline driven by a meta-agent:

1. The Digester: Compresses massive volumes of raw execution traces into structured summaries. This makes the data manageable for the downstream stages.
2. The Planner: Constructs an "adaptation landscape." It identifies which components are implicated in failures and proposes high-level directions for change.
3. The Evolver: Generates concrete, type-safe candidate edits. These include new Python tools or revised prompt templates.
4. The Critic: Assesses these candidates to prevent common RL pathologies.

The authors identify three specific risks in symbolic evolution. First is reward hacking, where the evolver tricks the verifier (e.g., by embedding answers directly into prompts) without actually solving the task. Second is catastrophic forgetting, where an edit fixes one problem but breaks a previously solved capability. Third is under-exploration, where the system gets stuck making trivial, low-risk changes instead of attempting major structural shifts.

To combat these, AEGIS employs a "deterministic gate." This enforces a "seesaw constraint": no candidate is accepted if it causes a regression in any previously successful task. To handle heterogeneous tasks, the authors introduce Ensemble Routing (or variant isolation). As shown in, this allows the system to maintain multiple harness variants.

It routes specific tasks to the version of the harness that has historically performed best for them.

Breaking the Scaffolding Ceiling

The most significant innovation is the concept of Harness-Model Co-Evolution. The authors argue that agents face two distinct bottlenecks. The "scaffolding ceiling" occurs when a fixed model can no longer exploit a better harness. The "training-signal ceiling" occurs when a model's new capabilities go unused because the harness never presents the right context.

By interleaving harness evolution with model training via a shared replay buffer, HarnessX attacks both simultaneously. As shown in, the system uses a method called Cross-Harness GRPO (Group Relative Policy Optimization).

This allows the model to receive gradient signals from "inter-strategy reward contrasts." Essentially, the model compares its performance across different versions of the harness. It learns to internalize the most successful strategies discovered during the evolution process.

The empirical results are striking. Across five diverse benchmarks, HarnessX achieved an average gain of +14.5% in task success rates. This means that, on average, the system adds nearly 15 percentage points of accuracy simply by optimizing the interface. Crucially, the researchers found an "inverse-scaling" pattern. The greatest gains were seen in weaker, smaller models. For example, on the ALFWorld benchmark, the Qwen3.5-9B model saw a massive +44.0% improvement. In contrast, the much stronger Claude Sonnet 4.6 gained only +11.2%. This suggests that a sophisticated harness can effectively compensate for a model's inherent reasoning limitations.

Implementation and Costs

For engineers considering this approach, the computational costs are notable. The evolution process requires significant upfront investment. For example, one GAIA run consumed approximately 143.7 million tokens for the global strategy. However, these costs can be amortized. On the GAIA benchmark, the evolved harness reduced per-task token consumption by roughly 25%. This happens because the system learns more targeted tool selection, leading to shorter trajectories.

Implications and Limits

This work suggests that agent progress is not solely a matter of scaling parameters. Instead, the runtime interface is a potent, actionable lever. We can now view agent development as a dual-track optimization problem. One track refines the model's weights. The other refines the discrete, symbolic architecture of its environment.

However, HarnessX has clear boundaries. The researchers note that the current framework is limited to discrete, text-based action spaces. It has not yet been tested in continuous environments like robotics. Additionally, the system currently relies on a closed-source meta-agent (like Claude Opus 4.6) to drive the evolution. It remains unclear if open-weight models can achieve the same level of sophisticated planning.

Finally, there is the issue of selection bias. Because the evolution occurs on the same task sets used for evaluation, the reported gains may reflect some degree of overfitting. Future work will need to explore whether these evolved harnesses can generalize to entirely unseen task distributions.

Figures from the paper