DeNovoSWE: Scaling Long-Horizon Environments for Generating Entire Repositories from Scratch

From Bug Fixer to Architect: Can AI Build Entire Software Projects?

Researchers have created a new way to train AI coding agents to build entire software projects from scratch rather than just fixing small bugs. They built a massive dataset of 4,818 tasks where the AI must read documentation and write all the code needed to make a working repository. This helps AI move from being a simple "editor" to a full "architect" of software.

While current Large Language Model (LLM) agents are increasingly capable of resolving individual programming issues, they struggle when asked to handle "long-horizon" tasks. These are complex sequences of actions that span extended periods and multiple files. Most existing training data focuses on localized bug fixing. This teaches an agent how to repair a single broken component but not how to design a cohesive system. The central question is whether we can move beyond "maintainer-like" edits toward "architect-level" construction.

The architectural bottleneck

The authors of the DeNovoSWE study investigate a fundamental gap in how we train autonomous coding agents. Currently, most software engineering (SWE) benchmarks, such as SWE-bench, focus on issue-level tasks. These tasks require an agent to find a bug in an existing codebase and apply a fix. While impressive, this doesn't simulate the reality of starting a new project from a high-level specification.

To truly automate software engineering, an agent needs to perform whole-repository generation. This requires long-horizon reasoning. The agent must plan how various modules, classes, and functions will interact across an entire file system. The authors argue that the primary obstacle isn't necessarily model intelligence. Instead, it is a lack of verifiable training data. Without thousands of examples of "documentation-to-code" transitions, models cannot learn the structural interdependence required to build a functioning repository from nothing.

The limits of localized training

Until now, the field has largely relied on scaling up datasets centered on single-issue fixes. This approach assumes that if an agent gets better at fixing bugs, it will eventually become better at building systems. However, the authors point out a significant crack in this logic. As shown in, prior datasets have focused almost exclusively on localized editing within existing codebases.

Even when researchers attempted to scale repository-level generation, they hit a ceiling of diversity and scale. Previous benchmarks like NL2Repo and BeyondSWE-Doc2Repo contained only 104 and 50 task instances, respectively. These small sample sizes meant that agents weren't being stressed by the variety of architectural patterns found in the real world. Furthermore, existing documentation on GitHub is often too messy or unstructured to serve as a reliable "blueprint" for training an AI. If the instructions are broken, the AI cannot learn to build a working house.

Synthesizing a digital blueprint

To solve the data scarcity problem, the researchers developed an automated, sandboxed pipeline to manufacture high-quality training instances. Instead of relying on humans to write documentation, they used a "divide-and-conquer" strategy. This was combined with an iterative "critic-repair" mechanism.

The process begins by decomposing a repository into distinct functional capabilities. Rather than asking an agent to understand the whole repo at once, the system profiles the code. It identifies which functions and classes are essential to the repository's core behavior. Once these pieces are identified, a multi-agent pipeline takes over. A "draft agent" writes initial documentation for a specific capability. A "critic agent" searches for omissions or technical inaccuracies. Finally, a "repair agent" fixes the flaws. This cycle repeats until the documentation is sufficiently detailed to act as a complete specification.

The authors used GPT-5.4 and GPT-5.5 to implement these agent modules and judge components. Crucially, they implemented a rigorous "leakage prevention" protocol. In a training environment, an agent might "cheat" by accessing the original source code through hidden caches or network calls. To ensure the task is a true "closed-book" test, the researchers strip away Git histories and purge temporary files. They also restrict network access within the sandbox. This ensures the agent's success comes from following the documentation, not exploiting the environment.

Breaking the 5% barrier

The impact of this new data is stark. The researchers fine-tuned a Qwen3-30B-A3B model on the DeNovoSWE dataset. They used a maximum context length (the amount of text the model can "read" at once) of 131,072 tokens. They tested it on the challenging BeyondSWE-Doc2Repo benchmark. Before training, the base model managed a meager 5.8% success rate. After fine-tuning on DeNovoSWE, the authors report that the score jumped to 47.2%.

The study also highlights a critical nuance in how we select training data. When dealing with complex, long-horizon tasks, a simple "pass-fail" filter is often counterproductive. If you only keep trajectories that pass 100% of unit tests, you might throw away the most informative examples. Even great agents struggle with very hard repositories.

The authors addressed this with a "difficulty-aware trajectory filtering" strategy. As illustrated in, as task difficulty increases, the average success rate of agents tends to drop.

By setting lower, adaptive thresholds for harder tasks, the researchers preserved a diverse range of high-quality reasoning paths. This allowed the model to learn from "partial successes" in complex environments. This prevents the model from being starved of data by overly strict requirements.

Shifting the engineering paradigm

The findings from DeNovoSWE suggest that the path to autonomous software engineering lies in the nature of the data. The authors demonstrate that training on repository-scale generation is more effective for long-horizon tasks than simply adding more issue-level bug-fixing data.

If these results hold as models scale, we may see a shift in how AI coding tools are developed. Instead of focusing on "copilots" that suggest the next line of code, the industry may move toward "architect agents." These agents could take a product requirement document and output a deployable repository. However, the paper does not explore whether these agents can handle extremely large-scale enterprise architectures. A logical next step would be to test if this "divide-and-conquer" documentation synthesis can scale to repositories with hundreds of interconnected microservices.

Figures from the paper