HERO Bootstraps Robot Skills From Zero Human Demos
Most manipulation stacks still start with human demos or teleop. HERO (Heuristic reasoning, Exemplar reuse, Reflexive execution under Orchestration) argues you can bootstrap and consolidate skills from zero human demonstrations by practicing in the loop.
Shanghai Jiao Tong University and University of Sussex authors posted arXiv:2607.26809 on July 29, 2026, with videos and figures at hero-agent.github.io.
Three capability layers
HERO is a hierarchical embodied agent:
- L1 — Heuristic Bootstrapper: zero-shot. A VLM decomposes instructions into four primitives — Grasp, Place, Push, Pull — grounded in a 3D workspace rebuilt with Depth-Anything-3 from multi-view RGB.
- L2 — Exemplar Accelerator: one-shot reuse. Similar past trajectories are retrieved and warped by rigid transform so the system skips full VLM replanning.
- L3 — Reflexive Execution: a closed-loop visuomotor policy distilled from successful practice, trained at 15 Hz with automatic fallback to L2/L1.
Tasks and numbers
Four real-world tasks appear on the project page: package picking, block stacking, drawer search, and cuboid discovery (each with reverse variants). Starting from zero human demos, HERO autonomously collects 664 forward and reverse subtask episodes. Recurring behaviors are distilled into L3 policies using 30 trajectories per task and 10,000 training steps. Deployment schedules L3 → L2 → L1 with up to 3 retries.
Code is listed as coming soon on the project site. The abstract frames the result as reduced human intervention during data collection while still hitting robust manipulation across those tasks.
A Human’s Take
The interesting claim is not “no humans forever” — it is practice as a first-class data engine. If L1 really fills the cold-start and L3 eats the routine, you spend human time on exceptions instead of every tray of demos. I want third-party replications before I trust the intervention counts outside the authors’ lab.