PARCEL: Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient Vision-Language Understanding

When we try to make Large Vision-Language Models (LVLMs) faster by using fewer visual details, they often get blurry or lose track of where objects are. Current methods usually pick one of two flawed paths. They either aggressively downsample the image, which acts like a low-pass filter (a process that removes high-frequency detail) that smears fine details. Alternatively, they replace the image with non-local summaries that destroy spatial grounding (the ability to link tokens to specific coordinates).

This paper introduces PARCEL. It aims to allow a single model to run at multiple visual-token budgets—ranging from 16 to 256 tokens—without the catastrophic loss of detail seen in previous "elastic" models. It essentially attempts to decouple the "where" from the "what" during the compression process.

The Problem

The status quo for elastic inference involves training a model that can handle variable token counts. This often uses Matryoshka-style nested structures. However, the authors identify a fundamental tension between the two primary ways to achieve this.

First, there is rigid spatial downsampling (exemplified by M3). This treats compression as a series of pooling operations. The authors argue that this behaves like an imperfect low-pass filter. This induces spectral aliasing (an artifact where high-frequency signals fold into lower frequencies). This is particularly damaging for tasks like ChartQA or document reasoning where pixel-level precision matters.

Second, there is query-only resampling (exemplified by MQT). Instead of a fixed grid, this uses learned queries to summarize the image. While this avoids the blurring of pooling, it sacrifices explicit spatial relationships. The queries become non-local summaries that lack a grid-aligned anchor. This significantly degrades the model's ability to perform dense spatial grounding. For example, it struggles to localize objects in the RefCOCO dataset. As shown in, these two approaches suffer from opposing spectral bottlenecks.

One loses high-frequency detail, while the other loses the structural "baseband" necessary for layout stability.

How It Works

PARCEL resolves this by partitioning the labor of feature extraction into two distinct pathways, a "division of labor" illustrated in .

Rather than forcing a single set of tokens to handle both geometry and semantics, the architecture uses a hybrid approach.

1. Spatial Anchoring: The model first applies budget-aware average pooling to create "2D Anchor" pool tokens. These tokens serve as low-frequency layout anchors. They secure the geometric structure of the scene.
2. Pool-Conditioned Query Resampling (PCQR): This is the core innovation. Instead of letting query tokens wander freely, PARCEL explicitly conditions them on the spatial anchors through a self-attention mechanism. This makes the queries "pool-aware." They understand the coarse layout before they attempt to find details.
3. Semantic Exploration: Once the queries are conditioned on the anchors, they perform cross-attention with the raw, uncompressed visual features from the Vision Transformer (ViT). These are the "Semantic Explorer" tokens. Because the anchors have already "claimed" the low-frequency layout, the queries focus on complementary, high-frequency visual signals.
4. Dynamic Routing: To maintain elasticity, the model uses a piecewise routing strategy. For low budgets ($16 \leq B < 64$), it uses a $4 \times 4$ anchor grid. For medium-to-high budgets ($64 \leq B \leq 256$), it scales up to an $8 \times 8$ grid. This ensures that as you increase the token budget, you also provide a richer spatial foundation.

The complete pipeline, from the SigLIP encoder to the Gemma-2 decoder, is mapped out in .

Numbers

The authors report that PARCEL significantly shifts the performance-efficiency Pareto frontier (the optimal trade-off curve between accuracy and speed). According to the results in, PARCEL consistently maintains higher mean retention across 27 benchmarks compared to M3 and MQT.

For engineers concerned with the cost of deployment, the paper provides theoretical FLOP (floating-point operations) and KV-cache (key-value cache) estimates. At a 256-token budget, PARCEL achieves a 95.1% mean retention rate for images. For video, it reaches 98.0% retention. In the video domain, where token counts scale with frame count, this efficiency is critical. The authors demonstrate that at 256 tokens, PARCEL outperforms MQT and M3. It keeps the KV-cache costs identical to other methods at matched budgets. The primary savings come from the reduced sequence length entering the LLM.

The impact on specialized tasks is telling. On the RefCOCO suite, which tests spatial grounding, PARCEL reaches 90.6% mean retention at a 256-token budget. This outperforms MQT by +4.2 points and M3 by +8.9 points. On resolution-sensitive tasks like ChartQA, the authors report gains of +4.7 and +3.4 points over M3. These gains occur at 64 and 256 tokens, respectively. This validates the effectiveness of the spectral decoupling shown in .

What's Missing

While the results are strong, there are a few gaps that a practitioner should note.

First, the budget selection is manual. The paper notes that the budget is specified by the practitioner. It is not predicted by the model based on input complexity. In production, you might prefer an input-adaptive mechanism. Such a system would decide to use 16 tokens for a simple landscape and 256 tokens for a dense spreadsheet.

Second, the training cost is significant. The authors mention that the PCQR block itself adds negligible overhead. However, the requirement for intermediary pretraining (100M samples) is a major investment. This is not a "plug-and-play" layer for a frozen model. It requires a dedicated training stage to align the new components.

Third, the paper focuses heavily on the PaliGemma-2 backbone. We do not see how it scales to much larger vision encoders or different LLM architectures. There is also no discussion regarding how this handles extremely high-resolution inputs beyond the high-resolution stress test in the appendix.

Should You Prototype This

Yes, if your deployment target involves variable compute constraints. This is useful when switching between a high-end GPU server and an edge device. The "train once, deploy anywhere" paradigm is ideal for heterogeneous fleets.

However, do not expect an immediate win if you want a training-free optimization. This is an architectural change. It requires a significant intermediary pretraining pass. If you have the compute budget for a 100M-sample pretraining stage, the performance gains in spatial grounding and video understanding make this a viable candidate. Code is reportedly available; see the paper for the canonical link.

Figures from the paper