Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

Can Smaller Models Outsmart Giants by Thinking Longer?

In the race to build autonomous AI agents—systems that plan, use tools, and solve complex problems—the prevailing wisdom has been to build bigger. Most frontier models rely on scaling parameters (the internal variables that store knowledge). This essentially adds more "neurons" to internalize vast reasoning patterns. But as tasks grow longer, a critical question emerges: is sheer size the only way to achieve intelligence? Or can we achieve similar performance by focusing on the complexity of the task's timeline?

Researchers have developed Agents-A1, a 35B Mixture-of-Experts (MoE) model. An MoE model uses a subset of its parameters for each task to remain efficient. This model is designed to tackle this exact tension. Instead of chasing trillion-parameter scales, the team focused on "scaling the agent horizon." This means increasing the length and complexity of the trajectories (sequences of actions and observations) the model learns from. The authors report that this approach allows a compact 35B model to match or even outperform massive 1T-parameter models on several long-horizon benchmarks.

The pursuit of the long-horizon agent

The central investigation asks whether an agent's competence can be scaled by increasing its "horizon" rather than its internal capacity. In AI agents, the "horizon" refers to the number of steps, tool calls, and observations a model navigates to reach a goal.

The authors aim to bridge the performance gap between mid-sized models and trillion-parameter giants. They do this by optimizing two specific dimensions. First, they scale the length of the trajectories the model learns from. Second, they scale the variety of specialized abilities the model integrates. The goal is to move toward models that can "do things" over extended, multi-step processes. Examples include conducting scientific research or performing complex machine learning engineering.

The limits of parameter scaling

Historically, the field has relied on the scaling laws of large language models. The assumption was that sophisticated reasoning requires more parameters to house that complexity. While effective, this route presents a significant barrier to entry. Reproducing the capabilities of a 1T-parameter model requires astronomical computational resources.

Furthermore, the authors identify a bottleneck in existing "horizon scaling" efforts. Previous attempts often lacked a unified way to connect external knowledge, actions, and feedback. Without a structured way to link observations (what the agent sees) with actions (what it does) and verification (whether it succeeded), agents struggle. They cannot easily recover from mistakes or learn from a long process. This lack of "process-level" supervision meant that agents often collapsed into one-shot lookups. They failed to develop true, iterative reasoning skills.

Building a knowledge-action infrastructure

To overcome these hurdles, the researchers developed a specialized training pipeline. This pipeline centers around a Knowledge-Action Graph (KAG). As illustrated in, the KAG acts as a structured memory of the agent's journey. Rather than just storing a final answer, the graph decomposes competence into five "atomic abilities." These are information acquisition, tool calling, executable iteration, evidence verification, and constraint tracking.

The KAG represents the agent's experience as a series of linked records. These records include the state of the world, the action taken, the resulting observation, and the verifier's verdict. This allows the model to learn from the entire "trace" of a task. To expand this data, the authors employed a "proposer–solver–verifier" game. In this cycle, one model proposes a task. Another attempts to solve it using tools. A third verifies the result. Only high-quality, verifiable trajectories are added to the graph.

The training follows a three-stage recipe shown in .

First, the model undergoes full-domain supervised fine-tuning (SFT) to gain broad behaviors. Second, the team trains specialized "teacher" models for specific domains. Finally, they use Domain-Routed On-Policy Distillation (OPD) to merge these experts into a single student model. To prevent conflicting reasoning styles, they use "Salient Vocabulary Alignment" (SVA). This ensures the student learns from the teacher's most relevant and high-probability suggestions.

Matching trillion-parameter performance

The results suggest that scaling the horizon is an effective alternative to scaling parameters. The authors report that Agents-A1 achieves leading results on several benchmarks. It outperforms 1T-parameter models like Kimi-K2.6 and DeepSeek-V4-pro on SEAL-0 (56.4), IFBench (80.6), HiPhO (46.4), FrontierScience-Olympiad (79.0), and MolBench-Bind (56.8). For a researcher, these scores mean the 35B model can compete with models 30 times its size in complex, multi-step tasks.

However, the researchers note where the model still faces challenges. While Agents-A1 excels in scientific and search-heavy tasks, it is weaker than 1T-level models on the MLE-Bench-Lite benchmark. The authors suggest this is because machine learning engineering requires maintaining a stable goal. It also requires avoiding repetitive, fruitless experiments over very long periods. This level of executive function remains a challenge for the 35B model.

A new path for agent development

These findings signal a shift in how we might develop specialized AI. Instead of the brute-force approach of building ever-larger models, developers could focus on richer "experience graphs." This would make high-performing agents more accessible to those without trillion-parameter budgets.

The immediate implication for practitioners is that "intelligence" in agents may depend on the quality of training trajectories. Raw model size may be less important than the complexity of the processes the model has mastered. For researchers, the work highlights a shift in supervision. We must move from "text-to-text" supervision to "action-to-observation" supervision. A logical next step involves testing this approach in real-time environments. This could include robotics or financial trading, where the "verifier" is a physical or market reality.

Figures from the paper