Do Text Edits Generalize to Visual Generation? Benchmarking Cross-Modal Knowledge Editing in UMMs

Do Text Edits Generalize to Visual Generation?

When we teach an AI a new fact using text, it often fails to show that new knowledge when it draws a picture. This paper introduces a new test called UNIKE to measure this gap. It also proposes a "reasoning" trick to help the AI better connect its new words to its new images.

Does textual knowledge imply visual truth?

The research focuses on Unified Multimodal Models (UMMs). These are systems that use a single, shared backbone (a central neural network architecture) to represent both images and text. This unification allows for "seamless parameter sharing." This means that updating a fact in the text-processing layers should theoretically update the image-generation capabilities.

The core question is whether text-side knowledge editing (KE) generalizes across modalities. KE is the process of modifying specific facts in a model's weights without full retraining. Specifically, if we edit a model to assert that "an apple is blue" in text, will it synthesize an image of a blue apple?

The assumption of seamless integration

Until now, developers assumed that unified architectures facilitate a natural transfer of knowledge. Previous work on knowledge editing has matured for text-only Large Language Models (LLMs). Researchers use methods like ROME or MEMIT to locate and alter specific neural weights. These methods update factual recall in text.

Because UMMs are built on joint representation, it was assumed that a successful textual edit would steer visual conditioning pathways (the processes that guide image synthesis). The field has largely operated under the impression that "unified" refers to a functional unity. In this view, knowledge resides in a shared conceptual space accessible to both linguistic and visual heads.

Testing the modality gap with UNIKE

To test this, the authors developed UNIKE. This is a benchmark with 2,971 edit subjects. These cover attribute edits (like color) and relation edits (like occupation) .

The investigation applies standard parameter-editing methods to models like Ovis-U1, BLIP3o-4B, and OmniGen2.

The researchers use VQA-based (Visual Question Answering) verification. They use a VLM (Vision-Language Model) to judge if the generated image matches the edited fact. This provides a more reliable metric than generic image quality scores.

The researchers implemented two protocols. The "DIRECT" protocol tests if the parameter edit alone changes the image. The "REASONING-AUGMENTED" protocol adds an intermediate step. The model is prompted to verbalize a rationale (an intermediate textual thought) before generating the image [Figure 1(b)]. This tests if explicit activation of the edited knowledge can bridge the modality gap.

A striking divergence in performance

The results reveal a massive discrepancy between textual success and visual realization. The authors report that text-side efficacy can reach approximately 92%. This means the model correctly answers text questions about the new fact. However, the best overall VQA accuracy under direct image generation is only 18.5%.

This "modality gap" is profound. It implies a model can be linguistically convinced of a fact while remaining anchored to original visual priors. Even with reasoning augmentation, the gap persists. Reasoning can improve VQA accuracy by up to 18.6 percentage points. This improvement helps, but it does not eliminate the disconnect.

Performance varies by category .

Attribute edits and relation edits fail in distinct ways. Also, as scene complexity increases, performance decays .

This includes moving from single objects to complex compositions or "derived products" (objects that inherit properties from the subject).

The conditioning-pathway bottleneck

The most significant finding is the "conditioning-pathway bottleneck." Mechanistic analysis suggests that edit-induced perturbations (small changes in neural activations) are often too weak to steer the Diffusion Transformer (DiT). The DiT is the component responsible for generating the actual image pixels.

In models like Ovis-U1, a frozen linear projection maps text representations to the visual generator. This projection acts as a filter [Table 4]. It preserves information for standard generation. However, it attenuates the "off-distribution" signals introduced by targeted parameter editing.

I read this as a warning regarding the "unified" label. Architectural unification does not guarantee epistemological unification. If this generalizes, it implies current editing research is incomplete. It currently ignores the specialized pathways required for visual grounding. We cannot treat a multimodal model as a single cohesive mind if its "beliefs" are trapped in the linguistic subsystem.

For practitioners, this means you should not rely on simple text-side edits for multimodal models. You should use reasoning-based prompting. This forces the model to verbalize the new fact. Doing so creates a stronger semantic constraint for the image generator. For researchers, the next step is to develop "modality-aware" editing methods. These should target the intersection of linguistic and visual conditioning pathways directly.

Figures from the paper