Count Anything

If you have ever watched a crowd-counting model fail when faced with a sudden shift in object scale, you know the pain. A model trained on large, sparse pedestrians often collapses when asked to count tiny, dense cells under a microscope. This happens because current counting research is fragmented. Models are usually specialists tied to specific visual domains, scales, or density distributions.

The Problem

Most existing counting models are bound to narrow scenarios. A model that excels at counting vehicles in remote sensing imagery will likely struggle with repetitive textures in agricultural crops. Current approaches generally fall into three buckets, each with a breaking point.

Density-estimation models predict a continuous heatmap (a grayscale map representing object concentration). They are effective for dense crowds but lack "instance-grounding." This means they cannot provide a one-to-one correspondence between a prediction and a specific object. Detection-based models provide clear locations but frequently miss or merge small, crowded, or weakly bounded objects. Point-regression models attempt to find discrete centers. However, they often lack the geometric context needed for large, sparse objects. As shown in, a generalist model must reconcile these conflicting requirements.

It must follow a natural-language query while providing spatial evidence for every count.

How It Works

The authors propose "Count Anything," a model that rejects the idea of a single counting representation. Instead, they implement a dual-granularity architecture. This design treats sparse and dense counting as complementary tasks.

1. Text-Conditioned Encoding: The model uses a backbone based on SAM3 (a pretrained segment-anything model) to extract visual features. These features are conditioned on the user's text query. This allows the model to focus on the specific semantic concept requested.
2. Region-level Sparse Counter (RSC): This branch uses a DETR-like (Detection Transformer) architecture. It predicts bounding boxes and their centers. It acts as an "anchor" logic. It provides stable geometric bounds for large, sparse, or clearly defined objects.
3. Pixel-level Dense Counter (PDC): The PDC handles cases where the RSC fails, such as tiny, overlapping cells. It operates on high-resolution features. It performs dense point-level enumeration. It predicts coordinate offsets relative to a spatial grid to capture small, weakly bounded targets.
4. Complementary Count Fusion (CCF): Both branches might predict the same object. To prevent this, the model uses a parameter-free fusion rule during inference. As illustrated in, if an RSC region and a PDC point overlap, the model compares their confidence scores.

It suppresses the lower-confidence prediction. This prevents the "double-counting" error found when simply unioning the two sets.

The model is trained using "point-centric supervision." This allows the researchers to ingest messy, real-world data. It unifies heterogeneous annotations—like bounding boxes, polygons, and masks—into a single training signal. Every valid instance is supervised by a central point.

Numbers

The authors evaluate their model on a massive new dataset called CLOC. This dataset contains 220K images and 15M instances across six domains. The headline result is a significant jump in accuracy. The paper reports a Mean Absolute Error (MAE) of 9.34 and a Root Mean Square Error (RMSE) of 33.34 on the CLOC test split. This substantially outperforms previous open-world methods like CLIP-Count or CountGD++.

For engineers concerned with deployment, the computational footprint is documented in [Table 7]. Under a 1008×1008 input resolution, the model has 848.62M parameters. It requires 5.44 TFLOPs per forward pass. On an RTX 4090, the authors report an average latency of 95.55 ms. Peak memory usage is 6.53 GB. This latency profile makes the model suitable for high-resolution batch processing or interactive inspection tools on modern workstation hardware.

What's Missing

There are gaps that a practitioner should note. First, the model relies heavily on a pretrained text-conditioned visual encoder (SAM3). The paper does not investigate behavior when text queries are highly ambiguous. If your use case involves esoteric scientific jargon, the "text-guided" aspect might become a bottleneck.

Second, the CLOC dataset is imbalanced. The authors admit that visual-domain scales are not perfectly balanced. General Scene and Remote Sensing dominate the dataset. This means the model's "generalist" capabilities might be biased toward these domains. This could lead to higher error rates in smaller domains like Microbiology or Agriculture.

Finally, the CCF fusion rule is a heuristic. The authors note in their ablation studies ([Table 10]) that the choice of neighbor group size is sensitive. In extremely crowded or occluded scenes, the current logic might still struggle to distinguish between a legitimate duplicate and two very closely adjacent instances.

Should You Prototype This

Yes, if your work involves high-precision counting in specialized domains like agriculture, pathology, or remote sensing. The ability to unify heterogeneous annotations into a single training pipeline is a massive operational win. This is particularly useful when dealing with "dirty" real-world data that lacks uniform formats.

The dual-granularity approach is a strong architectural bet for variable-scale objects. If you are currently struggling with a model that misses small objects in dense clusters, the RSC/PDC split is a pattern worth implementing. Code is reportedly available at count-anything. Start by testing it on your domain's "hard" cases. Look specifically for the scale transitions where single-paradigm models typically break.

Figures from the paper