Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models

Can We Build a Truly Unified Vision Model?

Real-time vision demands models that are accurate, efficient, and simple to deploy. This is a massive engineering hurdle for applications like autonomous driving and robotics. These systems often run on edge devices (hardware located near the data source, like drones or phones). Such devices have strict limits on power and processing speed.

Historically, the YOLO (You Only Look Once) family of models has been the industry standard. They strike a balance between speed and accuracy. However, even the best YOLO models face persistent bottlenecks. They often require a post-processing step called Non-Maximum Suppression (NMS) to clean up duplicate detections. NMS is an algorithm that removes overlapping boxes. They also use heavy mathematical modules that bloat their size. Finally, they struggle to "see" very small objects during training.

The authors of this paper aim to resolve these contradictions. They ask if it is possible to create a single, unified model family. This family would eliminate the need for NMS. It would shrink the detection head without losing accuracy. It would also handle everything from tiny objects to open-vocabulary text prompts within one pipeline.

Breaking the NMS and DFL bottleneck

The fundamental question the authors investigate is whether structural inefficiencies are actually necessary. Specifically, they target two major "taxes" paid during inference (the process of running a trained model) and training.

First, they look at the NMS dependency. Most CNN-based (Convolutional Neural Network) detectors rely on NMS. NMS is a post-processing algorithm. It looks at overlapping bounding boxes and suppresses all but the most confident ones. While effective, NMS adds latency (delay). It also complicates deployment on specialized hardware. Previous work like YOLOv10 attempted to remove NMS via dual-head training. However, the authors argue that prior attempts failed to balance the two training branches. This leaves the "inference-ready" branch under-optimized.

Second, they target Distribution Focal Loss (DFL). DFL is a technique used to refine bounding-box regression (the process of predicting box coordinates). It predicts a distribution over multiple "bins" rather than just four raw coordinates. While this improves localization, it massively inflates the number of parameters. This is a heavy burden for "nano" scale models meant for ultra-low-power devices.

The cracks in the current YOLO paradigm

Before YOLO26, the field operated under several assumptions. Researchers have been incrementally pushing the Pareto front. The Pareto front is the optimal boundary representing the best possible trade-off between accuracy (mAP) and speed (latency) .

However, the authors identify several cracks in this progression. One major crack is the "zero-assignment" problem for small objects. Current label-assignment strategies, such as Task-Aligned Learning (TAL), rely on geometric overlap. If an object is smaller than the model's smallest feature stride, it might not contain any "anchor centers." This means the model receives zero gradient signal (the mathematical instruction used to update the model) for that object. Essentially, the model is never taught that the tiny object exists.

Furthermore, the authors note an "optimization asymmetry" in dual-head models. This creates a mismatch. If you train a model with one head for dense supervision (to help it learn) and one head for sparse inference (to make it fast), you must balance them. If you treat them as equally important throughout training, the head used in production ends up being the weakest link.

A coordinated training intervention

The investigation proceeds through a coordinated suite of three training innovations. These are designed to compensate for a stripped-down architecture.

The first move was the removal of DFL. To prevent this from tanking accuracy, the authors implemented Progressive Loss. As illustrated in the training pipeline, this is a curriculum-style strategy.

Early in training, the model focuses on the "one-to-many" branch to build robust feature representations. As training progresses, the weight $\alpha(t)$ shifts toward the "one-to-one" branch. This ensures the head used during actual inference receives the most intense optimization pressure.

The second move addressed the small-object problem via STAL (Small-Target-Aware Label Assignment). Instead of using the actual tiny box for selecting candidates, STAL uses a "surrogate" box. This box is artificially enlarged to the nearest feature stride. This ensures that even a microscopic object triggers a positive assignment. This provides the gradient signal necessary for learning.

The third move involved the optimizer. The authors adapted MuSGD, a hybrid optimizer. It blends standard SGD with Muon. Muon is an optimizer used in Large Language Model (LLM) training. It utilizes orthogonalized updates (mathematical transformations that maintain perpendicularity) to improve conditioning. By applying this to high-dimensional weight tensors, they aimed to accelerate convergence (the speed at which a model learns).

Recovering accuracy through optimization

The findings suggest these training refinements do more than just mitigate the loss of DFL. They actually improve the model's ability to handle complex geometries.

The authors report that YOLO26 achieves a COCO mAP of 40.9–57.5. This is achieved at latencies of 1.7–11.8 ms on a T4 GPU . Crucially, the ablation studies in [Table 2] show that removing DFL initially drops accuracy. However, the combination of STAL, L1 loss, and backbone refinement fully recovers the baseline performance.

One of the more striking results is found in high-resolution tests. In [Table 3], the authors demonstrate that the DFL-free head performs better at 1280 resolution than at 640. They argue that DFL's finite range acts as a constraint on large objects. By moving to direct regression, the model can more reliably capture the full extent of large targets. This is qualitatively seen in .

In specialized tasks, the improvements are even more pronounced. For oriented bounding box (OBB) detection, a new angle loss was introduced. This loss is specifically for square objects. It helps resolve the ambiguity that occurs when a box's width and height are nearly equal .

Implications for the edge and beyond

The success of the DFL-free, NMS-free approach suggests that the "complexity tax" of modern models is avoidable. If you can achieve high accuracy with a lighter head, the barrier to deployment drops. This makes it easier to run high-performance vision on extremely cheap hardware.

The coordination between the optimizer (MuSGD) and the loss schedule (Progressive Loss) is also vital. It implies that the future of vision model training may look increasingly like LLM training. The crossover of techniques from the transformer world into the CNN world suggests a unified scaling recipe.

Finally, the YOLOE-26 extension shows that these efficiency gains port to open-vocabulary settings. Open-vocabulary refers to models that can detect objects not seen during training using text prompts. The paper shows that a stronger backbone allows for better zero-shot capabilities. This is evidenced by the LVIS results [Table 12].

Figures from the paper