Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models

Can We Train AI to Predict the Future Without Losing the Plot?

Researchers have developed a new way to make AI video generators much faster and more realistic. By combining two different training methods—one that uses perfect data and one that learns from its own mistakes—they can generate high-quality video in just one or two steps.

Current video generation models often struggle with a fundamental tension. They can either be highly accurate by looking at the whole video at once, or they can be fast and "streaming" by predicting one piece at a time. The latter approach, known as autoregressive generation, mimics how humans perceive time. However, it frequently falls victim to "exposure bias" (a training-inference gap). In this state, small errors in early frames compound. This causes the video to drift into nonsensical hallucinations.

The central challenge addressed by this work is how to bridge the gap between stable, offline training and the messy, unpredictable reality of real-time, on-policy generation.

Solving the Autoregressive Drift

The authors propose a new training recipe called Causal-rCM to stabilize autoregressive video diffusion. In this context, "autoregressive" means the model generates a video sequence piece-by-piece. It uses its own previously generated frames as context for the next. While this enables streaming, it creates a massive training-inference gap. During training, the model usually sees perfect, ground-truth history. During inference, it sees its own potentially flawed predictions.

The researchers suggest this gap can be bridged by leveraging a mathematical symmetry. They propose pairing two complementary forces. First, a "forward-divergence" force provides stable, offline training. Second, a "reverse-divergence" force provides on-policy refinement. This pairing aims to combine the reliability of ground-truth data with the realism of self-generated samples.

The Cracks in Current Refinement

Before this work, the field relied on fragmented strategies to fight error accumulation. Many existing systems use "teacher-forcing" (training with perfect history). This is stable, but it fails to prepare the model for its own imperfect outputs. Conversely, "self-forcing" (training on its own generated rollouts) attempts to fix this. While it addresses the training-inference gap, it is notoriously difficult to stabilize. It is often prone to "mode collapse" (a failure where the model repeats a limited set of safe, blurry patterns).

As illustrated in, existing paradigms like Diffusion-Forcing (DF) attempt a middle ground.

They add noise to the history, but they still do not perfectly simulate cumulative errors. The authors argue that the lack of a unified framework leaves researchers choosing between the stability of the teacher and the realism of the self.

A Two-Stage Divergence Strategy

The authors report that the solution lies in a sequential, three-stage pipeline. Instead of solving everything at once, they decouple the problem into two distinct mathematical objectives.

First, they use "Teacher-Forcing Consistency Models" (TF-CM). Think of this as a student studying with an answer key. The model is forced to stay consistent with the ground-truth trajectories of a powerful "teacher" model. This provides a stable, mode-covering initialization. Second, they apply "Self-Forcing Distribution Matching Distillation" (SF-DMD). This is the "on-policy" phase. Here, the model finally learns from its own generated videos to match the teacher's distribution.

To make this computationally feasible, the researchers developed a custom-mask FlashAttention-2 JVP kernel. This specialized software allows them to compute complex mathematical tangents (required for continuous-time consistency models) much more efficiently. The authors report that this infrastructure enables continuous-time models to achieve 10× faster convergence compared to discrete-time versions.

Achieving State-of-the-Art Speed

The results of this dual-force approach are significant. The study finds that TF-CM is currently the best way to initialize the more volatile SF-DMD stage. By providing a solid foundation of "correct" causal structures, the model is far less likely to collapse during refinement.

In terms of raw performance, the authors report that their distilled 2-step causal Wan2.1-1.3B model achieves a VBench-T2V score of 84.63. This represents state-of-the-art streaming video generation. Crucially, this high quality is achieved with only one or two sampling steps. As shown in, the Causal-rCM approach consistently outperforms previous methods like LongLive or AnyFlow.

Interestingly, the authors observe a nuance in how these models behave depending on how the video is sliced. In "frame-wise" settings (generating one frame at a time), they found that very shallow rollouts (1–2 steps) were actually more stable. These shallower steps helped prevent "camera drift" (uncontrolled rotation of the virtual camera). In "chunk-wise" settings (generating small groups of frames), deeper denoising (4 steps) was necessary. This was required to maintain motion consistency within the chunk, as shown in .

Implications for Interactive Worlds

If this recipe scales, it fundamentally changes the requirements for building interactive "world models." These are AI environments that can react to user inputs, much like a video game. The authors demonstrate this by applying Causal-rCM to Cosmos 3, an omnimodal foundation model. By replacing bidirectional attention with the causal attention used in their recipe, they transformed a static generator into an interactive simulator. This model can respond to action commands, such as steering a vehicle .

The immediate implication is that high-fidelity, real-time video streaming is no longer a strict trade-off between speed and stability. We can now theoretically have both. However, the paper notes that frame-wise training with long rollout depths remains somewhat fragile.

A logical next step for researchers would be to investigate whether a fully joint optimization could push the performance ceiling even higher. Currently, the authors use a staged pipeline because joint optimization proved challenging in causal settings.

Figures from the paper