Domain Arithmetic: One-Shot VLA Adaptation under Environmental Shifts

DART: One-Shot VLA Adaptation via Domain Arithmetic

When robots move to new environments, they often fail because their cameras or bodies have changed. A shift in camera pose or a transition from one robot arm to another (e.g., from a Panda to a UR5e) can render a high-performing policy useless. Currently, fixing this usually requires collecting massive amounts of new task-specific data for every single environment shift. This creates a severe deployment bottleneck.

This paper proposes a way to bypass that data tax. Instead of retraining from scratch, the authors suggest treating environmental shifts as mathematical vectors. By using a single demonstration, they propose a method called Domain ARiThmetic (DART). This method "subtracts" task-specific information and "adds" the new environment's characteristics directly into the model's weights.

The failure of one-shot fine-tuning

The status quo for Vision-Language-Action (VLA) models—policies that map visual observations and language instructions to robot actions—is either zero-shot deployment or heavy fine-tuning. Zero-shot often fails under environmental shifts. Meanwhile, full-data fine-tuning is too expensive to scale.

The logical middle ground is one-shot fine-tuning. This involves taking a single demonstration in the new environment and performing behavior cloning (a method where the model learns to mimic the demonstrator's actions). However, as the authors show in [Figure 2a], this approach fails to generalize. While the model might learn the specific task it was just shown, it loses its ability to perform other tasks in that new environment.

The authors' analysis in [Figure 2b] reveals why. The parameter updates ($\Delta$) produced by one-shot fine-tuning are overwhelmingly dominated by task-specific directions. There is a small domain-specific component present. Standard fine-tuning does not isolate it. It simply overwrites the base model's multi-task capabilities with task-specific behaviors.

Extracting the domain vector via analogy

DART treats adaptation as a weight-space analogy. It is similar to the "king - man + woman = queen" operation used in language model arithmetic. The core mechanism is to isolate a "domain vector" ($\delta_{tgt}$) that represents only the environmental shift.

The process follows these steps:

1. Dual Fine-tuning: The authors fine-tune the base policy ($\theta_0$) on a single demonstration in both the source domain ($D_{m,src}$) and the target domain ($D_{m,tgt}$) for the same task $m$. This produces two update vectors: $\Delta_{m,src}$ and $\Delta_{m,tgt}$.
2. Subtraction: By calculating $\delta_{tgt} = \Delta_{m,tgt} - \Delta_{m,src}$, the shared task-specific directions are theoretically canceled out. This leaves behind the delta caused by the environment.
3. Subspace Filtering: Direct subtraction is noisy. To clean the vector, the authors use Singular Value Decomposition (SVD, a method to break a matrix into simpler parts) to decompose the update vectors. They calculate an "overlap energy" for each source basis vector. This measures how well it aligns with the target subspace. They then discard source basis vectors that do not align. This prevents source-domain artifacts from corrupting the new domain vector .

4. Subspace Scaling: Finally, they scale the resulting vector by the subspace alignment score ($\gamma$). If the two updates are fundamentally misaligned, $\gamma$ will be low. The domain vector is then down-weighted to prevent injecting noise into the model.
5. Injection: The refined domain vector is added back to the base model: $\theta^* = \theta_0 + \alpha \cdot \tilde{\delta}_{tgt}$.

Evidence from simulation and reality

The authors report significant performance gains across various shifts. In simulation tests on the LIBERO benchmark using the $\pi$0.5 model, DART achieved an average success rate of 79.1% across novel viewpoints. This outperformed one-shot fine-tuning (31.5%) and existing methods like RETAIN and FLA [Table 1].

The method also scales to complex shifts. When combining viewpoint shifts with camera noise and illumination changes, DART maintained a 75.0% success rate. In contrast, standard fine-tuning dropped to 29.8% [Table 2]. The authors also demonstrated cross-embodiment transfer (moving from a Panda robot to a UR5e). They reported a 69.4% success rate on MimicGen tasks [Table 4].

The most notable evidence comes from the real-world UR10e setup. After adapting using only a single Stack Cube demonstration, the model's success rate rose from a zero-shot 43.3% to 81.7% across five different tasks [Table 5]. The ablation studies in show the method is robust to the scaling coefficient $\alpha$.

It maintains high performance across a wide range of values.

Practical caveats and blind spots

There are a few areas where a practitioner should be careful. First, performance degrades under "Large" viewpoint shifts. While DART still outperforms baselines in these extreme cases, the drop in success rate suggests a single demonstration may lack sufficient geometric information.

Second, there is a hyperparameter dependency. The authors acknowledge that a small search for the scaling coefficient $\alpha$ is required for optimal performance. In a production pipeline, this means you cannot simply "plug and play" without a quick validation step.

Third, the method assumes you can access or replicate a source-domain demonstration for the same task. The authors argue this is a "mild" assumption. Most VLA training sets are well-indexed. However, it could be difficult in messy, real-world datasets. If you cannot find a matching source task, you must rely on similarity-based retrieval. The authors note this is less effective [Table 22].

The verdict

DART is a step forward for deploying VLAs in non-static environments. It moves away from the "collect thousands of demos for every camera angle" paradigm. Instead, it offers a modular, algebraic approach to robot adaptation. The ability to merge multiple domain vectors into a single consolidated vector [Table 7] is useful. This can reduce memory overhead in robot fleets.

If your environment is guaranteed to be static, stick to standard fine-tuning. But if you are shipping robots into homes or factories, DART is worth a prototype. Lighting, camera mounts, and hardware iterations are constant. The code is available at https://github.com/snumprlab/dart.

Figures from the paper