Xiaomi-GUI-0 Technical Report

Closing the Gap Between Benchmarks and Real-World Mobile GUI Agent Usability

Most AI assistants designed to navigate smartphones are trained on simulations. These do not match the messy reality of daily life. While these models perform impressively in controlled labs, they often stumble in the real world. They struggle with sudden login prompts, payment verifications, or network errors. The Xiaomi-GUI-0 technical report proposes a solution. It uses a system trained and evaluated within a closed loop of actual physical devices. This ensures the AI learns to handle real-world friction rather than just perfect simulators.

The Problem of the Synthetic Sandbox

A Graphical User Interface (GUI) agent is an AI system that perceives a screen. It acts much like a human does. It executes actions such as tapping, swiping, or typing to complete a task. Unlike traditional software, GUI agents do not use APIs (Application Programming Interfaces, which are structured backend channels for data exchange). Instead, they interact directly with the visual layer. This makes them theoretically universal. They can use any app without needing special backend access.

However, a significant gap exists between laboratory performance and real-world utility. Most current agents are trained on "offline trajectories" (recorded sequences of successful actions). They often use emulated environments (virtualized software versions of mobile operating systems). These environments are too "clean." They lack the chaotic "abnormal states" encountered by real users.

In a real mobile environment, an agent might face a CAPTCHA (a security challenge to distinguish humans from bots). It might encounter a permission dialog for camera access. It could hit a risk-control mechanism that freezes an account. Because these states are rare in training data, models often fail to recognize them. This leads to repetitive, useless loops of clicking the same button.

Bridging Reality Through Hybrid Infrastructure

To move beyond the sandbox, the authors developed a "real-device-dominant hybrid infrastructure." As shown in, the system does not rely solely on physical hardware. Physical hardware is difficult to scale. It also does not rely solely on emulators. Emulators lack realism. Instead, it uses a tiered approach. Physical smartphones, tablets, and even in-vehicle cockpits serve as the primary execution substrate. Virtual sandboxes provide auxiliary support for scalable, reproducible data collection.

This infrastructure enables a sophisticated data collection strategy. The researchers constructed three distinct tiers of training data: 1. High-frequency task data: This targets common, well-defined operations like "adding to cart." It includes realistic abnormal states like login expirations. 2. High-generalization data: This targets "long-tail" intents (rare, complex requests). It uses a "function tree" and "behavior buckets" to generate diverse, realistic queries.

1. Agent-capability enhancement data: This bolsters high-level reasoning. It uses structured Chain-of-Thought (CoT) annotations. This forces the model to explicitly output its [Observation], [Reflection], [Plan], [Decision], and [Memory] at every step.

Crucially, the authors introduce an "error-driven data flywheel." Traditional learning cycles focus on scaling successful examples. This flywheel focuses on repairing failures. As illustrated in, human annotators can replay a failed trajectory.

They identify the "first key error" and provide a correction. Furthermore, a "teacher model" monitors the student agent. If the student’s performance drops, the teacher "takes over" .

The teacher demonstrates how to recover from the error and return to a successful path.

A Progressive Curriculum for Mastery

The training of Xiaomi-GUI-0 follows a three-stage pipeline. It moves from simple imitation to complex, autonomous decision-making.

First, Supervised Fine-Tuning (SFT) establishes the foundational "language" of the interface. It teaches the model basic action protocols and the structured CoT schema. Next, the researchers implement Step-level Reinforcement Learning (Step RL). This stage uses a "cascade reward" system to optimize individual responses. The model is judged on a hierarchy of requirements. First, is the command formatted correctly? Second, is the reasoning logically consistent with the action? Only then does the model receive higher-level rewards based on its actual capability.

Finally, the model undergoes Agentic Reinforcement Learning (Agentic RL). This is the most advanced stage. The model is optimized over entire, multi-step trajectories .

Here, the reward is not about a single tap. It is about whether the agent successfully navigated a long-horizon task. This includes maintaining "memory" (tracking task state across multiple pages) throughout the journey.

Measuring Success in the Wild

To prove this approach works, the authors introduced the RealMobile benchmark. RealMobile is built from real user traffic. It is executed on physical devices. It employs a fine-grained scoring system. Instead of a binary "success" or "failure," agents receive partial credit for completing specific sub-goals.

The results show a stark difference between simulated excellence and real-world competence. While Xiaomi-GUI-0 remains competitive on standard grounding benchmarks (locating icons on a screen), its true strength is in navigation. On the simulated AndroidWorld benchmark, the model achieved a 78.9% success rate.

More importantly, on the real-device RealMobile benchmark, it achieved a 72.0% success rate. This represents a massive jump in reliability for real-world use. For comparison, the authors report that Xiaomi-GUI-0 substantially outperformed the open-source model MAI-UI-8B. That model achieved only a 33% success rate on RealMobile. It also exceeded the performance of major proprietary models like Claude Opus 4.6 (33%) and Gemini 3.1 Flash (58%) in this specific real-world setting. This suggests that training on real-world failure modes is essential for deploying reliable agents.

Limits of the Loop

The Xiaomi-GUI-0 framework is not a panacea. The effectiveness of the "error-driven flywheel" depends on the quality of the teacher model. It also depends on the diversity of the initial error distribution. If the teacher model lacks reasoning depth, the student will not learn to recover. Additionally, the reliance on physical devices creates logistical bottlenecks. This limits data throughput compared to pure simulation. Future work must address how to maintain this realism as mobile ecosystems grow more complex.