SIMPLE: PSI Lab’s Dual-Sim Benchmark for Humanoid Loco-Manipulation
Humanoid foundation models are shipping faster than anyone can fairly compare them. USC’s Physical Superintelligence (PSI) Lab answers with SIMPLE — Simulation-Based Policy Learning and Evaluation for humanoid loco-manipulation — a dual-simulator testbed with 60 whole-body tasks, 50 indoor scenes, and 1,000+ object assets.
Dual engines on purpose
SIMPLE decouples physics from vision. MuJoCo owns contact-rich dynamics and high-frequency control; Isaac Sim owns photorealistic rendering after offline replay. The project page describes three stages: collect trajectories in MuJoCo (motion planning or VR teleop), replay and domain-randomize in Isaac Sim, then evaluate policies under three progressive randomization levels (L0 / L1 / L2) via a Gym-style interface.
Data collection is built in. Automated planning uses BoDex grasps and CuRobo dual-arm trajectories with a scripted base policy. VR teleop streams egocentric stereo to a PICO headset while a whole-body tracker holds balance. The authors report collection rates (demos/hour) on whole-body pick-and-place of 58.9 (motion planning), 206.8 (real teleop), and 310.3 (sim teleop) — sim teleop wins because resets and safety overhead disappear.
Who wins the leaderboard
The team fine-tunes and scores mainstream policies — including Ψ0, GR00T N1.6, π0.5, DreamZero, ACT, and others — on six task families under L0/L1/L2. Tables on the project site show Ψ0 and ACT among the strongest overall, with mobile pick-and-place still hard for several large VLAs. Ablations favor mixing domain-randomization levels and prefer teleop data over motion-planning-only for several task families.
Critically, policies trained only in SIMPLE transfer zero-shot to physical humanoids under matched settings. Reported sim vs real success includes pick-and-place 0.90 / 0.80 and handover 1.00 / 0.80. Code is open on GitHub (physical-superintelligence-lab/SIMPLE); paper: arXiv:2606.08278.
A Human’s Take
I love a benchmark that admits humanoid loco-manipulation is not tabletop arms with legs glued on. Dual-sim is the right trade: MuJoCo for feet, Isaac for cameras. The zero-shot numbers are the real pitch — if outside labs can reproduce them, SIMPLE becomes the default place to fail cheaply before you burn robot-hours.