Flexion, Niantic Spatial, and NVIDIA Close a Sim2Real Loop for Humanoids
Flexion and Niantic Spatial published a joint research write-up on July 20, 2026 showing an end-to-end path from a phone-or-360 capture of a real site to a humanoid navigation policy that runs on the real robot without extra fine-tuning. Training runs inside NVIDIA Isaac Sim / Isaac Lab on photorealistic Gaussian-splat reconstructions. That is a sharper story than another teleop farm announcement.
The pipeline in plain language
From Flexion’s research post:
- Capture — walk a deployment site once with an off-the-shelf 360° camera (no LiDAR survey required)
- Reconstruct — Niantic Spatial builds a metric-scale digital twin: photorealistic 3D Gaussian splat for RGB views plus a collision mesh derived from the same reconstruction (including multi-view stereo via MVSAnywhere)
- Export — package as a NuRec-compatible USDZ for Isaac Sim / Isaac Lab
- Train — Flexion trains an RGB local-navigation policy with reinforcement learning inside that twin, then deploys to hardware
The hard constraint they call out: visual geometry and collision geometry must agree. A few centimeters of mismatch teaches the policy to walk through walls it can see or dodge walls that are not there.
What they measured
Flexion focuses on local navigation — short-horizon motion from the robot’s current pose to a nearby goal using camera + proprioception + a goal in the robot frame, with a separate locomotion policy turning velocity commands into walking.
Simulation ablations (1,024 rollouts per condition) compare depth vs RGB and generic meshes vs site reconstructions. On two office reconstructions, RGB trained in the 3DGS twin was the only RGB setup that matched or beat a conventional depth baseline:
- Flexion office (easier): 97.8% success (RGB/3DGS) vs 93.8% (depth baseline)
- Niantic Spatial office (harder): 75.0% vs 70.9%
They also show the reconstruction-trained RGB policy transferring zero-shot to a real office robot and handling scene edits like moved furniture. Hard stereo-depth cases they highlight: semantic hazards (e.g. a blue mat that barely shows in depth), thin structures (tripods, cables), and glass.
Why this sits next to the teleop debate
In a separate July 19 Robot Report essay, Flexion CEO Nikita Rudin (ex-NVIDIA Isaac Gym / Isaac Lab) argues teleoperation datasets are orders of magnitude smaller than language-model corpora and cannot keep up with real-world variation. His company pitch — also laid out in Flexion’s Series A post ($50M from DST, NVentures, and others) — is a command / motion / control stack trained primarily with simulation and RL rather than permanent operator farms.
The Niantic work is the concrete demo of that worldview: specialize a policy inside a faithful twin of the target building, then ship it.
A Human’s Take
Zero-shot transfer from a splat twin is the kind of claim I respect when the evaluation numbers are published next to the video. These are still office navigation demos, not full factory manipulation shifts — Flexion is honest that the next steps are semantics, scene variants, and long-horizon Reflect-style stacks. For multi-site fleets, “walk the building once, train overnight, deploy” is the product shape that actually scales. I’ll watch whether third parties can reproduce the depth-vs-RGB tables outside Zurich and London.