Most AI agents act like people who can only do one thing at a time and wait idly for a response.
In the current landscape of LLM-based agents, we treat tool use as a synchronous, sequential loop. The agent calls a function, the environment returns a result, and the agent proceeds. This works in a vacuum. However, it ignores the reality of distributed systems. In production, function calls incur non-negligible latency (delay). This includes network round trips, database queries, or heavy computation.
If an agent is tasked with multiple independent goals, the naive approach is to execute them one by one. This wastes the "idle time" spent waiting for a tool response. Such waste kills overall system throughput. The research behind AsyncTool addresses this. It tests whether agents can actually multitask. Specifically, it checks if they can utilize waiting intervals to advance other tasks.
The Problem
The status quo in agent evaluation is fundamentally flawed. It overlooks the temporal dimension (the timing and duration of events) of tool invocation. Most existing benchmarks, such as BFCLv3 or NESTFUL, operate in single-task settings. In these settings, tool responses are treated as instantaneous. Even when they allow for multi-step processes, they rarely simulate the asynchronous nature of real-world APIs.
When you introduce latency, the complexity increases. Current agents struggle to maintain state (the history and context of a task) while waiting. Instead of switching to a different task to stay productive, models often fail. They fall into modes like "tool confusion" (applying a tool meant for Task A to Task B, as seen in ). They also show "insufficient temporal awareness." This is where the model hallucinates a result because it refuses to wait for the actual tool response . Essentially, the industry measures intelligence in a world with zero latency. We do not have that luxury in the prod path.
How It Works
To capture this, the authors developed AsyncTool. This is a benchmark for evaluating asynchronous tool calling in interactive, multi-task environments. The core system is a simulated environment that injects realistic response latencies. This forces the agent to make a choice: wait or switch.
The researchers built the benchmark using a four-stage data evolution pipeline .
They did not just scrape data. They performed "Coarse Reconstruction" using Gemini 2.5 Pro. This produced strictly ordered, dependency-aware tool-call trajectories (the specific sequence of actions taken). They then used intensive human annotation to ensure logical consistency. This ensures that while tasks can be pursued concurrently, the internal steps of a single task must respect intra-task dependencies.
The interaction paradigm, shown in, treats the agent as a concurrent system.
For example, if Task 1 requires a slow file-system operation, a capable agent should detect the "wait" state. It should then immediately pivot to Task 2. The evaluation is granular and uses three levels: 1. Step Level: Basic F1 scores (a measure of precision and recall) for tool selection and parameter correctness. 2. Sub-task Level: Whether the agent completed the specific sequence and reached the correct environment state. 3. Task Level: End-to-end success across all interleaved tasks.
Numbers
The results confirm that asynchronous execution causes significant performance degradation. The authors report that even top-tier models see their scores tank when forced to deal with delayed feedback.
The standout performer is GPT-4.1. It achieved an overall task accuracy of 38.06. This represents the highest recorded success rate in these complex, interleaved scenarios. The paper notes a stark divide between closed-source and open-source models. For instance, while closed-source models generally maintain higher efficiency, smaller open-source models suffer from extreme degradation. Their sub-task completion rates drop sharply as more tasks are added.
One vital metric is the "Same-task Streak." This measures how many consecutive turns a model spends on a single task. The authors use this to map the trade-off between accuracy and efficiency .
A high-performing agent should ideally sit in the lower-right quadrant. It needs high overall accuracy and a low streak. This means it is actively interleaving tasks rather than grinding through one at a time. Frequent switching alone is not a silver bullet. Models that switch too much without proper dependency tracking end up in "tool confusion."
What's Missing
The paper is rigorous, but there are gaps. First, the dataset is bounded by its lineage. Because tasks are reconstructed from BFCLv3 and NESTFUL, the tool variety is limited. If your production agent uses highly specialized APIs, this benchmark might miss those edge cases.
Second, the construction process is expensive. The authors admit that relying on Gemini 2.5 Pro and manual human verification makes scaling difficult. For a practitioner, this means you cannot easily spin up a custom version of AsyncTool. Doing so requires significant compute and human overhead.
Finally, the paper focuses on the logic of switching. It stays relatively quiet on the architectural implications of an asynchronous agent loop. It tells us that agents fail at temporal reasoning. However, it does not deeply explore whether specialized architectures are the definitive solution.
Should You Prototype This
Yes, but with caveats. If you build agentic workflows that interface with real-world APIs, you cannot rely on standard single-turn prompting. You must implement explicit state management and dependency tracking.
Do not try to build a "better LLM" alone. Instead, build a better orchestration layer (the system that manages task flow). This layer should handle "wait" states. Use the principles in this paper to instrument your own internal benchmarks. If you cannot measure your agent's "Same-task Streak," you will not know why throughput is stalling. The code is reportedly available at https://github.com/StoKou/repo-asynctool. Start by prototyping a supervisor pattern. Manage these async transitions before you ship a monolithic agent to your users.