FlatLab Teaches Arms to Pick Up Things That Have No Handle
Books, boards, and towels lie there daring a parallel gripper to find a lip. A Jilin University group, with Peking, CAS, and Birmingham, built FlatLab so that problem has a shared sim and more than one trick.
The platform sits on NVIDIA Isaac Sim 4.5.0. The paper and project page list more than 100 physically simulated flat objects, 18 scenes in six room types, and two 7-DoF Franka arms. Code is promised on the project site.
Three strategies, then primitives
No single pre-grasp covers the set. Edge-push fails on large or floppy things. Dual-arm lift wants thickness. So the framework first classifies the object, then runs primitives.
- Strategy A: slide to the table edge (thin, small: disks, pads)
- Strategy B: two-arm lift (thick, large: boxes, paintings)
- Strategy C: squeeze an edge into a fold (towels, fabric)
The strategy generator reads a point cloud, uses simulated material/scale transforms, and contrastive learning so same-strategy objects cluster even when they look different. The execution module then sequences touch, slide, and squeeze primitives from scene point clouds, with separate losses for position and quaternion geodesic error.
The scoreboard
104 objects in 21 categories split into Train, Test α (unseen objects, known categories), and Test β (unseen categories). Strategy accuracy: 99.2% / 91.3% / 78.6%. Grasp success, five random orientations each, object held aloft two seconds: 81.1% / 74.2% / 69.0%.
They generated 1,390 object clouds for the classifier and 50 pose demos per object for execution. Single-strategy Slide and Lift baselines, plus Diffusion Policy, 3D Diffusion Policy, OpenVLA, π0, and π0.5, all land lower on the same protocol. The comparison table’s top baseline in the paper is 68.6% on train against their 81.1%.
On a real Baxter with a RealSense D455 and SAM for tabletop masks, they report 83.6% / 80.0% / 80.0% on the same three splits over 30+ household flats. The paper notes reality can be kinder than sim: real rigid friction holds, and real cloth actually wrinkles.
Accepted to ICML 2026. Limits they list: most work is still sim, scene clutter is untested, and they have not stuffed a giant VLA into the pipeline.
A Human’s Take
Picking a magazine off a table is the unglamorous cousin of humanoid laundry demos, and I am glad someone made a 100-object bench for it. Strategy first, primitives second is how you avoid teaching a diffusion policy to memorize one particular book. The Baxter numbers being higher than Isaac is a nice reminder that cloth in a warehouse renderer is still a little fake.