Moebius: 0.2B Lightweight Image Inpainting Framework with 10B-Level Performance

Researchers have developed a tiny but powerful AI tool for fixing images by filling in missing parts. Even though it is much smaller than the giant models used by big companies, it works just as well and is over 15 times faster.

In the current landscape of generative AI, image inpainting—the task of reconstructing missing regions of an image—is dominated by massive, generalist foundation models. These industrial-grade giants, such as FLUX.1-Fill-Dev, use tens of billions of parameters to achieve impressive zero-shot results. However, this scale comes at a steep price. Exorbitant computational costs and massive memory footprints make them nearly impossible to deploy on edge devices or in latency-sensitive production environments. The central question facing the field is whether we can move away from the "bigger is better" scaling paradigm. Can we build highly optimized, task-specific specialists that can bridge the performance gap?

The representation bottleneck of extreme compression

Current attempts to shrink inpainting models usually follow a predictable, and often failing, path. The standard approach is to take an existing architecture and swap out heavy components for lightweight alternatives. Examples include Depthwise Convolutions (DWConv) or linear attention.

The authors of the Moebius paper argue that this naive substitution triggers a severe representation bottleneck. Inpainting requires intense semantic reasoning and precise spatial-texture alignment. Striating away the heavy lifting of standard transformers causes a catastrophic degradation in quality. For example, while Gated Linear Attention (GLA) is efficient for self-attention, it lacks the mathematical formulation necessary to perform cross-attention. Cross-attention is essential for integrating external semantic priors (additional information about the image content). As seen in, when generalist models like SD3.5 or FLUX fail, they often manifest as color discrepancies, blurriness, or structural confusion.

The authors' goal was to compress the model without losing the ability to reason about these complex structures.

Reconstructing the backbone with LλMI

To solve the bottleneck, the researchers introduced the Local-$\lambda$ Mix Interaction (L$\lambda$MI) block. Instead of relying on memory-intensive dot-product attention—which scales quadratically with the number of tokens—they summarize information into fixed-size linear matrices, referred to as $\lambda$.

The L$\lambda$MI block functions through three distinct stages:

1. Local-$\lambda$ Module: This acts as a self-attention equivalent. It aggregates intra-image semantic and spatial contexts. This allows the network to maintain local spatial continuity without the quadratic cost.
2. Interactive-$\lambda$ Module: This solves the cross-attention problem. It allows the latent representations to interact with global semantic priors. Specifically, it uses Latent Categories Guidance (LCG) embeddings, which are semantic descriptions of the image categories. These are summarized into a compact matrix. This ensures the model can still ingest high-level category information.
3. Mix-FFN: To reach extreme levels of compactness, the authors replaced standard Feed-Forward Networks (FFN) with a depthwise-augmented structure called Mix-FFN. This significantly reduces the parameter budget.

By cascading these modules, the L$\lambda$MI block replaces the cumbersome spatial transformer blocks found in heavy diffusion models, as illustrated in .

Matching 10B-level performance at 0.2B

The effectiveness of this architecture relies heavily on an adaptive multi-granularity distillation strategy. Because extreme compression naturally weakens a model's capacity, the authors use a teacher-student framework to recover that lost intelligence. They perform this distillation strictly within the latent space. This avoids the massive overhead of decoding back to pixel space during training.

The results are striking. The paper reports that Moebius achieves a total inference acceleration of $>15\times$ compared to FLUX.1-Fill-Dev. While FLUX.1-Fill-Dev uses 11.9B parameters, Moebius operates with just 0.22B. This is less than 2% of the parameter count. Despite this, the authors demonstrate that Moebius rivals or even surpasses the generation quality of these industrial giants. On the CelebA-HQ portrait benchmark, Moebius achieves an FID (a metric where lower is better for image quality) of 5.39. This vastly outperforms the 10B-level industrial models.

Furthermore, the authors conducted a double-blind human preference study. As shown in, Moebius (31.76% preference) matched the performance of its 862M-parameter teacher (32.18%).

It also significantly outperformed the 10B-level generalists like FLUX.1-Fill-Dev (23.70%).

The cost of being a specialist

While Moebius is a powerhouse for targeted tasks, it is not a magic bullet. There are clear trade-offs to this extreme structural compression.

First, the model is a specialist. Its performance is tied to the benchmarks and domains it was trained on. The authors admit in their failure case analysis that Moebius may exhibit minor detail loss. It may produce less plausible textures in extremely tiny background regions. This happens when contextual information is severely limited compared to the larger 1B-parameter teacher.

Second, the "synergy" requirement is high. The paper's ablation studies show that you cannot simply drop these modules into a standard backbone. The success of Moebius depends on the tight coupling between the L$\lambda$MI architecture and the specific multi-granularity distillation loss. If you attempt to implement the L$\lambda$MI blocks without the accompanying adaptive gradient-based balancing, you will likely hit the representation bottleneck.

The verdict

If you are looking to deploy high-fidelity inpainting on resource-constrained hardware, Moebius is a definitive "yes." It proves that for specific tasks, a distilled 0.2B specialist can effectively compete with a 10B generalist.

The architecture is highly optimized. However, it is not a "plug-and-play" replacement for standard transformers. You cannot just swap modules. You must adopt the entire distillation-heavy training recipe to realize the gains. For engineers building dedicated inpainting services, this is a blueprint for moving toward efficient, on-device execution.

Code is reportedly available; see the project page at https://hustvl.github.io/Moebius for the canonical link.