RAI’s Koala Gripper: Dual-Thumb Hands for Humans and Robots
The Robotics and AI Institute put a paper on arXiv on August 20 for Koala, a gripper that exists in two bodies at once: a handheld capture device a person can run, and a motorized robot device that is supposed to copy it.
The point is not another five-finger humanoid hand. It is a dual thumb (the koala joke) plus two underactuated fingers, three controllable degrees of freedom, built so the camera view and the kinematics match from demo to replay.
Why co-design, not a bolt-on handle
Most handheld data tools start from an existing robot gripper and add a handle. RAI’s May 12 lab blog says that path failed on ergonomics and wrecked data quality. Koala was drawn as one mechanism, then split into a hand-powered trigger version and a motorized version.
From the paper and the project page:
- two 1-DoF preshaping underactuated fingers opposite a 1-DoF pivoting dual thumb
- a 9-bar finger linkage, one actuated DoF and one underactuated
- robot fingers described as backdrivable, with effective mass “on the order of tens of grams”
- three standard sizes and padded grips on the capture device
- the robot side uses a frameless motor and a high-pitch ballscrew for low reflected inertia
The blog is blunt about parallel jaws: practiced operators still struggled to run a ratchet, turn a screwdriver, or hold a nail while the other hand hammered. Koala is aimed at power grasps, pinch grasps, and human tools.
What they showed it doing
The project page lists grasps parallel jaws tend to lose: nails, keys, pencil pinches, cards, plates, bowls, cups. The paper’s abstract adds secure grasps on a wide object set, forceful tool use, and precise singulation, then an end-to-end imitation-learning pipeline from those demos.
I am not inventing success rates. The abstract does not print a percentage. The claim on the page is qualitative plus the hardware photos.
A Human’s Take
I am here for the boring matching problem. If the camera on the human handle does not see what the robot camera will see, the policy is guessing. Koala treats that as a mechanical design job, not a dataset job.
Three controllable DoFs will not fold laundry. It might run a drill without teaching a 20-DoF tendon hand first. That is a reasonable trade, if the imitation pipeline actually transfers. The paper says it does. I want the failure cases next.