ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation

Can an AI truly grasp the concept of "how many"? While modern vision-language models can describe a sunset, they famously struggle with basic arithmetic applied to pixels. Researchers have developed models that can count objects. Others can generate images from text. However, these two capabilities rarely live in the same brain.

A new study introduces ABACUS, a unified model designed to bridge this divide. It aims to create a single system. This system can both accurately count objects in a crowded scene and generate new images with the exact number of items requested. By treating counting and generation as two sides of the same coin, the authors suggest we can move past models that "know" what an apple looks like but "forget" how to count them.

The synergy gap in numerical vision

The central question driving this research is whether visual understanding and visual generation can be unified into a single, mutually reinforcing loop. In the current landscape, these tasks are treated as separate disciplines. Object counting involves estimating the number of instances in an image. This is often done by regressing a density map (a spatial representation where pixel intensity correlates to object presence). Conversely, count-conditioned generation requires synthesizing a scene that respects a specific cardinality (the total number of elements in a set).

The authors investigate why existing unified models fail to bridge these regimes. They identify a "synergy gap." A model might correctly count four apples in a photo. Yet, it fails to generate exactly four apples when asked. This represents a fundamental lack of spatial reasoning. The researchers hypothesize that the failure stems from a lack of spatial grounding during training. The model learns to recognize categories but fails to master "objectness"—the ability to perceive a distinct, individual entity even when surrounded by visually similar neighbors.

Cracks in the specialist approach

Before ABACUS, the field relied on two diverging paths. "Specialist" models were built for specific tasks. These include crowd counting or object detection. These models are highly accurate but brittle. A model trained to count pedestrians in a subway station typically cannot count apples on a table. Generalist vision-language models (VLMs) attempted to handle everything. However, they often collapsed when faced with fine-grained, instance-level instructions.

As shown in, these generalist models suffer from several distinct failure modes.

Text-to-image diffusion models often lack any internal mechanism to verify if their output matches the requested count. Meanwhile, existing multimodal large language models (MLLMs) tend to default to coarse magnitude estimates. They might say "there are many people" rather than "there are 42 people," especially in dense scenes. Furthermore, even when a model can perform both tasks, the two branches often work at cross-purposes. As illustrated in [Figure 2(c)], the same model that correctly counts four apples in an image often fails to generate exactly four from a text prompt.

Closing the loop with self-critique

To solve this, the authors implement a series of interconnected technical moves. They use a 3B-parameter foundation model. First, to fix the "dense scene" problem, they introduce density-aware adaptive zooming. As shown in, if a region is identified as dense, the model recursively partitions it into smaller $2 \times 2$ sub-regions.

This allows the model to operate on locally sparse fields. Per-instance discrimination is more reliable in these areas.

To ensure the model doesn't lose track of objects at the edges of these new crops, the researchers developed a boundary-aware count policy. This policy uses Group Relative Policy Optimization (GRPO). GRPO is a reinforcement learning method that optimizes a policy by comparing a group of potential actions against one another. The policy teaches the model how to assign "ownership" of an object that straddles a crop boundary .

The most significant move is the "cycle-consistent GRPO" strategy. Instead of relying on an external human or a separate model to grade the AI, the authors turn the model's own understanding branch into a critic for its generation branch .

The generation branch produces a group of candidate images. The frozen understanding branch then counts the objects in those images. It provides a reward based on how well the count matches the original text prompt. This creates a closed loop. The generator learns directly from its own mistakes.

Superiority across seven benchmarks

The results of this integrated approach are significant. The authors report that ABACUS achieves state-of-the-art results across seven different benchmarks. It outperforms both task-specific specialists and much larger generalist models. In object counting tasks on the FSC-147 dataset, the study finds that ABACUS achieves a test Mean Absolute Error (MAE) of 5.03. This passes the strongest specialist model (CountGD++) by over 40% without needing point-level annotations.

In the realm of generation, the improvements are equally stark. On the CoCoCount benchmark, the authors report that ABACUS achieves 71% exact-match accuracy using a YOLOv9 detector. This surpasses the previous best method (CountGen) by 21 percentage points [Table 3]. Crucially, the model maintains high visual quality. Some previous methods tried to force accuracy by manipulating attention mechanisms. This often resulted in distorted, "glitchy" images. In contrast, ABACUS achieves a significantly higher aesthetic score of 89 on GenEval, compared to only 61 for the baseline UniLIP-3B [Table 3]. This suggests the generation and understanding branches are optimized to work in harmony .

From counting to spatial reasoning

The success of ABACUS suggests a shift in how we might approach multimodal intelligence. If this cycle-consistent reinforcement learning generalizes, it implies we do not necessarily need massive, externally annotated datasets. Instead, we can use a model's inherent ability to "understand" to supervise its ability to "create."

There are two immediate implications for the field. First, it demonstrates that counting and generation are not competing objectives. They are actually mutually reinforcing. The act of generating forces the model to develop better spatial awareness. This, in turn, makes it a better counter. Second, it provides a blueprint for "self-correcting" AI. A model can refine its own outputs through internal verification loops.

However, the paper does not explore whether this logic extends to more complex spatial relationships. Examples include "the red ball is behind the blue cube." While the model excels at cardinality, it remains to be seen if this cycle-consistency can bridge the gap for full relational reasoning. A logical next step for researchers would be to test if this self-critique framework can improve performance on tasks involving complex geometric arrangements or relative positioning.

Figures from the paper