Robust-WAM Keeps Video Pretraining and Adds Semantic Foresight
World-action models that ride video generation pretraining are strong on dynamics and soft on appearance shifts. Flip the design into a pure semantic latent and you gain lighting robustness — but you lose the web-scale VAE video prior. Robust-WAM (arXiv 2608.05903) is a post-training recipe that keeps the VAE video path and injects semantic foresight into the action stream.
The trick
On top of an existing video-generation WAM:
- Leave the Video DiT denoising VAE latents alone (pretraining stays)
- Prepend learnable query tokens to the action DiT stream
- Align each query’s output to the frozen DINOv3 CLS embedding of the matching future ground-truth frame
- Reuse the positional encoding of the action step that reaches that future frame
Teacher and alignment head are training-only. At inference, only the queries remain.
Numbers that matter
From the paper and project page (success rates %):
LIBERO-Plus (train on standard LIBERO, test under camera/lighting/background/layout/noise axes):
| Base WAM | Clean LIBERO | LIBERO-Plus | With Robust-WAM (Plus) |
|---|---|---|---|
| FastWAM | 97.6 → 97.9 | 49.7 | 58.9 (+9.2) |
| GE-Act | 96.5 → 97.3 | 78.0 | 80.9 (+2.9) |
On FastWAM, large Plus gains include sensor noise (+18.9), camera (+12.4), lighting (+11.3), and background (+10.3).
RoboTwin clean→random (LingBot-VA unified WAM): random success 29.8% → 34.4% (+4.6) with clean held at 81.2%.
Real Franka (train under white light; evaluate purple/cyan only): GE-Act OOD average 57.3% → 80.0% with Robust-WAM; InD roughly preserved (80.0% → 82.7% on the reported averages).
A Human’s Take
Lighting OOD is where a lot of “it worked in the demo” dies. Robust-WAM’s bet is correct for operators: don’t throw away the video prior, post-train the action head to care about scene semantics. I’ll trust this more when third parties re-run the Franka purple-light protocol. Until then, the LIBERO-Plus jumps on FastWAM are the clearest receipt in the paper.