XPolicyLab Wants One Adapter Per Robot Policy, Not One Per Benchmark
Connecting N robot policies to M benchmarks still costs O(NM) custom glue. Different VLAs assume different camera names, gripper scales, and Python stacks, so every leaderboard rebuilds the same mess.
XPolicyLab, submitted to arXiv on Aug 10, 2026 (arXiv:2608.09892) by contributors led from MMLab@HKU and Tsinghua, ships a shared contract that drops that cost to O(N+M). One adapter per policy. One client per environment. The same adapters already serve RoboTwin, RoboDojo simulation, and RoboDojo-RealEval.
What they built
The paper specifies common observation, action, and trajectory schemas plus a minimal adapter interface: update_obs, get_action, batched variants, and episode reset. A dependency-isolated client/server bridge (WebSocket + MessagePack) keeps policy inference in its native conda/world while the simulator or robot driver stays in its own stack, local or remote.
As of August 2026 the ecosystem lists 42 policies — OpenVLA-OFT, π0 / π0.5, GR00T-N1.7, Diffusion Policy, ACT, several world-action models, and more. Model-specific adapter code varies by an order of magnitude; the environment-facing loop stays within a few lines of a fixed reference.
Numbers that matter
In a controlled study (N=6 engineers, within-subject, Cursor + Opus 5 for the agent condition), integrating π0.5 into RoboDojo simulation took:
- From scratch: more than 5 hours, ~300 hand-written LoC
- XPolicyLab manual: ~2 hours, ~120 LoC
- XPolicyLab + agent skills: ~30 minutes, ~0 hand-written LoC
Reproducing an already-integrated policy is about 10 minutes excluding checkpoint download. Snapshot leaderboards (Aug 2026) show FastWAM at 77.8% mean clean success on RoboTwin and π0.5 leading RoboDojo-RealEval score (22.9 / 12.8% SR) — the point is not the rankings, it is that every entry shared one observation contract.
Code and site: github.com/XPolicyLab/XPolicyLab, xpolicylab.github.io.
A Human’s Take
I’m so here for boring plumbing that makes leaderboards comparable. The paper is honest that absolute real-robot scores are still low and that sim ranks only partially track metal. What changes is the cost of getting a checkpoint to run correctly somewhere it has never run. That is the gate that burns lab weeks. If the agent-skill path holds outside the authors’ checkout, open policy work gets a real shared socket instead of another README full of environment-specific tips.