FAPO: Fully Autonomous Prompt Optimization of Multi-Step LLM Pipelines

Modern AI workflows rarely consist of a single question and a single answer. Instead, they rely on multi-step LLM pipelines. These are complex sequences that combine retrieval, reasoning, and formatting steps. Because these steps interact, a failure in a middle stage can cascade. This causes the entire system to collapse.

Currently, most developers tackle this by manually tweaking individual prompts. This is like trying to fix a broken assembly line by only adjusting worker instructions. You might ignore the fact that the conveyor belt moves too slowly. If the bottleneck is structural (relating to the arrangement of steps), no amount of clever prompting will solve it.

A new study introduces FAPO (Fully Autonomous Prompt Optimization). This framework moves beyond mere prompt tuning. Instead of just fixing one prompt at a time, FAPO uses an AI agent to look at an entire multi-step workflow. It identifies exactly where a process is failing. It distinguishes between a bad search step or a formatting error. It then automatically fixes the prompts or the structure of the workflow to improve performance.

The limitations of prompt-only tuning

Traditional optimization methods, such as GEPA, focus on searching for better instruction strings within a fixed program. While effective for single-turn tasks, the authors argue this approach misses critical bottlenecks in multi-step chains. When a pipeline fails, the error might not be a misunderstanding of instructions. It might be a lack of sufficient information (a retrieval issue). It could also be a failure to follow a specific output schema (a formatting issue).

Existing tools primarily optimize the "textual" interface of a model. Consequently, they are blind to the underlying mechanics of the pipeline. If a reasoning step fails because the preceding retrieval step did not fetch the right documents, a prompt optimizer will struggle. It might simply try to tell the model to "guess better." This is a fundamentally flawed strategy. To truly optimize a complex system, the optimizer must see the "why" behind a failure. It needs to understand the "what" of the incorrect output.

An agentic loop for structural repair

FAPO approaches optimization as an autonomous engineering task. The framework utilizes Claude Code as an orchestrator to drive a continuous, closed-loop process. Unlike a static optimizer, FAPO operates within a standardized, isolated workspace for each task. This ensures that changes are reproducible. It also prevents data leakage (the accidental sharing of sensitive info) between different projects.

The mechanism functions through several specialized agents working in concert, as illustrated in .

First, the system evaluates the current pipeline variant on a set of training cases. Second, a step-attribution subagent analyzes the intermediate outputs. These are the "logs" of what happened at every stage of the chain. The subagent classifies failures. It determines if a mistake was prompt-addressable or structural.

Once a failure is diagnosed, the optimization agent proposes a "scoped" change. FAPO follows a strict hierarchy. It always attempts prompt edits first. Only when the attribution report suggests that prompt-level search has plateaued does the system escalate. It may then change chain parameters or the actual pipeline architecture. To prevent the agent from breaking the system, a third agent acts as a safety gate. This variant-reviewer checks every proposal for scope compliance and potential data leakage before testing .

Escalation drives significant gains

The authors report that FAPO outperforms the GEPA baseline in 15 of 18 model–benchmark comparisons. The most striking evidence of FAPO's effectiveness appears when the system is permitted to move beyond text edits. In the six comparisons involving the HoVer and IFBench benchmarks, FAPO won every single one. In these cases, FAPO escalated to structural changes. It achieved a mean gain of +33.8 percentage points (pp). This represents a massive jump in accuracy compared to the baseline.

These gains represent fundamental shifts in capability. For example, on the HoVer task (many-hop fact verification), FAPO identified insufficient retrieval coverage. It responded by extending the baseline 3-hop retrieval chain to a 4–5 hop chain. Similarly, on IFBench (verifiable instruction following), FAPO added deterministic post-processing nodes. These nodes enforce formatting constraints.

Even when restricted to prompt-only optimization, FAPO remains competitive. In security-focused tasks like CTIBench-RCM, FAPO lifts test accuracy. It achieves gains of up to +7.1 pp on specialized models. As seen in, the optimization trajectory typically shows steady climbs in validation performance.

The agent iterates through increasingly sophisticated prompt variants to achieve these results.

Path dependence and mathematical noise

Despite these successes, the paper highlights important trade-offs. One notable drawback is that FAPO exhibits higher run-to-run variation. This happens when it is allowed to escalate to structural changes. The authors attribute this to "path dependence." This means that once the optimizer decides to change the architecture, it enters a different optimization landscape. This makes the outcome of a single run harder to predict.

Furthermore, the results on the AIME mathematical benchmark were inconclusive. In all three model comparisons for AIME, GEPA actually led FAPO. The authors speculate this may be due to overfitting (optimizing too closely to a specific dataset) on small sample sizes. This is particularly relevant in the vast problem space of competitive mathematics. For practitioners, this means performance on pure, high-complexity reasoning tasks may remain volatile.

A new standard for pipeline engineering

FAPO represents a shift from "prompt engineering" to "pipeline engineering." It treats the LLM workflow as an inspectable, editable piece of software. This provides a path toward much more reliable autonomous systems. The ability to automatically diagnose failures is a significant step. It reduces the manual toil involved in deploying LLM agents.

If you are building complex, multi-step workflows where reliability is paramount, FAPO is a tool worth prototyping. The code is reportedly available at https://github.com/cisco-foundation-ai/fully-automated-prompt-optimization. However, users should be prepared for increased computational costs. They should also expect higher variance when allowing an agent to rewrite application logic.

Figures from the paper