Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks

Instead of just using AI to search for solutions, researchers have found a way to teach the AI how to "evolve" them. By training models on millions of successful search attempts, they can impart complex problem-solving skills directly into the model's weights. This allows smaller, open-source models to learn sophisticated discovery patterns. These patterns were previously only achievable by massive, expensive proprietary models.

The ephemeral nature of search

In the field of optimization—the science of finding the best solution among a vast sea of possibilities—Large Language Models (LLMs) have become powerful tools. Researchers often combine LLMs with evolutionary search scaffolds (external frameworks that manage the search process). In this setup, the LLM acts as a "mutation operator." It proposes small changes to a candidate solution, such as a piece of code or a mathematical formula, to see if they improve the result.

However, the authors of this study identify a fundamental inefficiency in this status quo. Currently, the "intelligence" required to navigate a search resides almost entirely in the external scaffold. This includes knowing which parts of a solution to mutate or when to backtrack after a failure. Because the model's weights remain frozen during these searches, every new problem is approached from scratch. Once a task is finished, the expertise gained during that specific search is discarded. This creates a heavy reliance on massive, proprietary models. Smaller open-source models struggle to follow complex evolutionary trajectories without this external help.

Internalizing the evolutionary path

To bridge this gap, the researchers introduce Evolution Fine-Tuning (EFT). This is a "mid-training" paradigm designed to move discovery capabilities from the scaffold into the model. Rather than teaching a model to simply map a problem to an answer, EFT teaches the model to map an evolutionary state to a successful next step.

The process begins with the construction of the $\mathcal{F}$inch Collection. This is a massive dataset of 156,731 evolutionary trajectories. The authors built this using a three-step pipeline: 1. Seed Collection: They gathered 371 optimization tasks across 10 domains. These ranged from GPU kernel design to biological denoising. 2. Trajectory Generation: They used a high-capacity "teacher" model (Qwen3.5-397B-A17B) to generate sequences of solutions. Each entry records the transition from a "parent" solution to a "child" solution. This includes the prompt, the history of previous attempts, and the feedback from an evaluator. 3. Rigorous Filtering: To ensure the model doesn't learn bad habits, the authors removed problematic trajectories. They discarded entries with systematic errors, such as timeouts or syntax failures. They also removed "breakage" cases where a perfect parent resulted in a broken child .

The researchers then fine-tuned open-source models ranging from 2B to 9B parameters using these trajectories. Crucially, they don't just use successful transitions. They also utilize preference learning (specifically the KTO algorithm) to help the model distinguish between improvements and regressions. This teaches the model not just what a good mutation looks like, but also how to recognize a bad one.

Generalization across domains

The effectiveness of this approach is evidenced by the model's ability to transfer skills between unrelated fields. The authors report that $\mathcal{F}$inch models outperform their base counterparts by an average of 10.22% across 22 held-out tasks. This means the fine-tuning provides a significant boost in solving new, unseen optimization problems.

Perhaps more striking is the emergence of cross-domain strategy transfer. For example, when tasked with a competitive programming problem, $\mathcal{F}$inch does not rely on a single, repetitive strategy. Instead, it pulls from its diverse training history. It can apply techniques learned in recommender systems or numerical optimization to solve algorithmic challenges .

The study also demonstrates clear scaling laws. Increasing the number of training tasks in the $\mathcal{F}$inch Collection from 15 to 355 leads to a 14.1% average improvement in held-out performance . Furthermore, the researchers show that EFT acts as an effective "warm-up" for test-time Reinforcement Learning (RL). When paired with RL scaffolds, $\mathcal{F}$inch achieves state-of-the-art performance on two circle-packing tasks. It also exceeds the performance of its base model on the Erdős minimum-overlap problem.

Limits of the discovery agent

While the results are significant, the paper does not claim to have created a universal solver. There are several technical boundaries to consider. First, the researchers acknowledge a potential limitation regarding search scaffolds. Because they trained exclusively using the OpenEvolve scaffold, it is unclear if $\mathcal{F}$inch will generalize to different architectures like EvoX.

Second, the synergy between EFT and test-time RL was primarily demonstrated on mathematical tasks. The authors note they have yet to verify if this combination provides the same boost for engineering tasks. This includes optimizing GPU kernels, which might involve different types of feedback loops. Finally, the current framework is limited to "single-turn" mutations. This means the model looks at a state and proposes one change. The authors suggest that moving toward multi-turn reasoning is a necessary next step for true autonomy.

The verdict

If you are looking to deploy capable optimization agents on local hardware, $\mathcal{F}$inch is a compelling proof of concept. It reduces the need for massive, expensive proprietary APIs. The research proves that "process-based" training is a viable way to imbue small models with expert-level intuition. This is done by learning from the journey of discovery rather than just the final destination.

The approach is ready for experimentation in specialized domains. However, it is not yet a "plug-and-play" replacement for frontier models in all engineering contexts. For practitioners, the immediate value lies in the $\mathcal{F}$inch Collection itself. It provides a scalable blueprint for turning raw search data into reusable model intelligence. Code and models are reportedly available; see the paper for the canonical links.

Figures from the paper