1× speed · Fully autonomous · Zero-shot sim-to-real
10 object categories, each shown with 3 videos of 3 distinct object instances.
Human hand-object interactions (HOIs) provide a rich source of demonstrations for dexterous manipulation, but learning directly from them presents challenges in bridging morphology gaps, ensuring dynamical feasibility, and sim-to-real deployment. We present Morphometric Imitation, a three-stage framework that transforms reconstructed HOIs into zero-shot sim-to-real visuomotor policies. First, morphometric optimization (MMO) kinematically retargets human motion across hand morphologies while preserving demonstrated contacts. Second, residual reinforcement learning (RL) refines the kinematic reference using object pose and contact information from the human motion to produce dynamically feasible robot demonstrations. Third, these demonstrations are distilled into visuomotor policies. Across three robot hands and ten HOIs, MMO improves contact F1 over the strongest of five baselines by at least 8 points for every hand, while also improving the success rate of downstream dynamic retargeting by as much as 35 points. Ablations on the residual RL show complementary benefits from using object pose and contact information. Finally, the visuomotor policies achieve 89.3% zero-shot success in 300 real-world trials on 30 objects.
Explore MANO, morphed hands, object grasps, and robot motion across three hand–object pairs.
Drag to rotate and inspect the hands. Scroll to zoom · On touchscreens, drag with one finger and pinch to zoom.
Loading the interactive scene…
Compare the MANO hand motion, kinematic retargeting, and RL result for each object across three robot hands.
One human motion, retargeted to all three robot hands.
Hammer
One human motion, retargeted to all three robot hands.
Apple
One human motion, retargeted to all three robot hands.
Cube
Select an object above. Each clip loops at its original speed; use the video controls to pause or inspect a moment.
@misc{sadjadpour2026morphometricimitationmorphologycontact,
title={Morphometric Imitation: From Morphology and Contact Aware Hand Retargeting to Sim-to-Real Visuomotor Policy},
author={Tara Sadjadpour and Siming He and C. K. Wolfe and Haozhi Qi and Lea Wilken and S. Shankar Sastry and Claire Tomlin and Jitendra Malik},
year={2026},
eprint={2609.28660},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2609.28660}
}