In-Context VLA: Consume Grounded Language, Don’t Narrate CoT
Most generalist manipulators clone expert action chunks from an image and a fixed instruction. The tempting upgrade is free-form chain-of-thought text. In-Context VLA, posted August 6, 2026 (arXiv:2608.05738), argues that path often degrades low-level control: ungrounded reasoning, latency that breaks closed-loop timing, and conflicting objectives that train the policy to narrate instead of act.
Consume language, don’t emit a novel
The authors’ fix: what a VLA needs is the ability to consume grounded language, not to generate essays. Their framework uses:
- In-context post-training — perceptual evidence is injected as structured context; supervision stays on actions only.
- Agentic tool-use — the policy queries open-vocabulary detectors, monocular depth, and a vision-language model to fetch task-relevant facts on demand.
A data engine produces diverse, paraphrased, evidence-conditioned spatial descriptions so the policy learns to interpret language it has never seen verbatim — rather than one templated caption per scene.
Where it scored
Across RoboCasa-GR1, SimplerEnv, and LIBERO simulation benchmarks, plus 8 real-world robot manipulation tasks, the paper reports SOTA performance and efficiency versus CoT-based approaches under matched configurations. The teaser figure shows in-context learning at 0.97 success rate with 1.00 normalized cost efficiency bars against a CoT baseline at 0.88 SR and 0.22 on their cost metric — illustration of the paper’s efficiency claim, not a substitute for full tables.
A Human’s Take
This matches what I’ve felt watching CoT demos on arms: the robot pauses to monologue, then jerks. Forcing the model to read tool-backed spatial context while only scoring on actions is the sane division of labor. If tool calls stay cheap and the paraphrases stay noisy, this is a pattern other VLA stacks should copy before they bolt on longer token chains.