EgoPHI Reads 3D Hand Force From a Single Head-Cam Frame
ETH Zürich’s Sensing, Interaction & Perception Lab posted EgoPHI on August 13 (accepted to ECCV 2026). It is, the authors say, the first method that jointly estimates dense contact maps and 3D force on hand and object meshes from one monocular RGB image plus object geometry.
Authors: Andela Ilic, Rachel Schuchert, Yijing Jiang, Christian Holz. Code is listed at github.com/eth-siplab/EgoPHI.
Contact is not force
Prior vision work (PressureVision, EgoPressure) mostly maps pressure on flat surfaces. EgoPHI targets articulated, non-planar objects. It registers estimated hand meshes, refines object pose in the camera frame, then predicts per-vertex contact and 3D force.
There is almost no scalable force ground truth, so they simulate it. Recorded hand–object meshes from ARCTIC go into SOFA. Soft virtual springs fire when vertices enter a contact zone; force is −k d n with k = 10. A contact mask then drops spurious non-contact forces.
Numbers they report
Trained only on ARCTIC, EgoPHI beats a fine-tuned HACO baseline on force: hand MAE drops from 6.62 N to 4.03 N (RMSE 7.29 → 5.06). On out-of-distribution H2O, it still produces object contact and force estimates; HACO does not. Iterative pose refinement cuts mean per-vertex pose error by more than 70% (ARCTIC 64 → 19 cm, H2O 48 → 10 cm). Those centimeter errors tell you pose is still the weak link.
For sim-to-real, they built a cube and a cylinder from 8 mm acrylic with an LED strip and a fisheye camera underneath. Light leaking into a finger is their force proxy. Eight participants, about 2,000 synchronized egocentric/contact pairs. SOFA vs this FTIR setup: cylinder MAE 0.36 ± 0.18 N (Spearman 0.760), cube 0.25 ± 0.16 N (0.604).
They list the gaps: no temporal model, object mesh required, only normal forces, uniform stiffness, and the two real objects are still simple shapes.
A Human’s Take
If you can get a usable 3D force field from a GoPro-style view, robot learning-from-demo gets a lot less tactile-sensor-shaped. I would not hang a grasp policy on 4 N MAE yet. The useful result is the transfer: simulated springs that line up with an acrylic light trick. Next test is a mug or a pair of pliers, not another cube.