One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding

Current AI agents often need to "zoom in" on a screen twice to find exactly where to click. This makes them slow. This secondary pass is a common workaround for Multimodal Large Language Models (MLLMs) that struggle with pixel-perfect precision. This precision is required for GUI grounding—the task of mapping natural language instructions to executable coordinates.

Until now, the industry standard has been a "crop-and-rerun" strategy. The model predicts a coarse region first. Then, the system crops and enlarges that area. Finally, a second forward pass executes to find the exact coordinate. While effective, this doubles the latency and computational overhead. This is a major hurdle for real-time interactive agents. A new paper, One Forward Beats Two, argues that this second pass is unnecessary. The authors claim the model already "sees" the target in its intermediate layers. It simply fails to carry that evidence to the final output.

The Region-to-Point Gap

The authors identify a structural bottleneck called the "Region-to-Point Gap." In typical MLLM-based grounding, the model uses autoregressive coordinate generation (predicting $x$ and $y$ tokens one by one). Through diagnostic analysis, the researchers found that target-region awareness emerges early in the decoder process. As shown in, "ROI Recall" (how well the model's internal attention maps cover the ground-truth target) peaks in the intermediate layers.

This peak occurs between layers 19 and 23.

However, this spatial awareness collapses by the final layer. The authors report that ROI Recall drops by an average of 54.9 percentage points from its peak to the final layer. Even when the model identifies the right region, it often fails to produce an accurate click. This mismatch means the model knows where the button is, but loses the fine-grained visual cues needed for precision. Existing "ZoomIn" methods solve this by brute-forcing a second look at a cropped region. This is an expensive way to fix a signal preservation problem.

Bridging Evidence via Internal Zoom

Instead of an external crop, the authors propose InnerZoom. This framework reuses internal target-region evidence within a single forward pass. The architecture, outlined in, operates through three stages:

1. Target-Region Extraction: The model uses its own intermediate text-to-vision response maps to create a coarse target-region proposal. It identifies high-response connected components in the attention heatmap. It then pulls fine-grained, unmerged visual features from the vision encoder.
2. Cross-Layer Evidence Refinement: This is the core of the method. The authors introduce an "Iterative Dual-Slot Evidence Adapter." This shared adapter maintains two learnable "slots"—a focus slot and a context slot. These act as a compact workspace. As the forward pass moves through selected decoder layers, these slots iteratively refine the extracted visual evidence. This prevents the signal from fading.
3. Evidence-Guided Decoding: To prevent the refined evidence from being diluted, the model uses a KV-only injection strategy. It injects the refined evidence directly into the key/value projections (the parts of the model that store information for attention) of the target-region positions. The query projections remain unchanged. This keeps the original decoding logic intact while sharpening the "memory" of the target region.

Efficiency Gains Without the Second Pass

The most compelling argument for InnerZoom is the efficiency delta. The authors show you can achieve higher accuracy than two-pass methods while cutting costs. Under a controlled 4B parameter setting, the paper reports that InnerZoom outperforms the two-pass ZoomIn baseline by 1.3 points on average.

For production deployment, InnerZoom reduces end-to-end latency by up to 31.8%. It also reduces TFLOPs (total floating-point operations, a measure of computational work) by about 29% compared to ZoomIn. illustrates this trade-off.

InnerZoom delivers higher accuracy with a much smaller computational footprint. On the OSWorld-G benchmark, the InnerZoom-4B model achieved a score of 64.7. This surpassed previous best results by 4.1 points. The scalability of the approach is also supported by results on 2B models. These showed similar improvements over the base SFT+RL (Supervised Fine-Tuning + Reinforcement Learning) baselines.

Blind Spots in Localized Evidence

While the results are strong, there are technical edges to consider. First, InnerZoom depends on the quality of the initial intermediate response. If the model's initial attention is heavily biased, the refinement pipeline may zoom in on the wrong area. The authors note that since region selection is only indirectly optimized, robustness in very noisy environments remains a challenge.

Second, the method might not replace explicit visual re-observation for all tasks. For extreme cases, like a dual-screen setup, an explicit crop might provide higher raw pixel density. This density could be difficult to recover in a single pass. Finally, the qualitative analysis in highlights persistent failures. These include "Type I" errors (semantically complex instructions) and "Type III" errors (distractions from on-screen text). This suggests the bottleneck sometimes shifts from spatial precision back to linguistic reasoning.

The Verdict

InnerZoom is a significant step toward making GUI agents viable for real-time use. By treating grounding as a signal-preservation problem, the authors bypass the heavy tax of test-time scaling.

If you are building an agent that requires low-latency interaction, this is worth a prototype. The move from a two-pass "look-then-act" loop to a single-pass "refine-while-acting" architecture is an efficient design. It delivers better performance by working smarter with the data already flowing through the weights. Code and models are reportedly available; see the paper for the canonical link.

Figures from the paper