A Modular Approach to the Embodiment of Hand Motions from Human Demonstrations

Alexander Fabisch, Manuela Uliano, Dennis Marschner, Melvin Laux, Johannes Brust, Marco Controzzi

In: Proceedings of the IEEE-RAS International Conference on Humanoid Robots. IEEE-RAS International Conference on Humanoid Robots (Humanoids-2022) November 28-30 Ginowan Japan IEEE 2023.


Manipulating objects with robotic hands is a complicated task. Not only the fingers of the hand, but also the pose of the robot’s end effector need to be coordinated. Using human demonstrations of movements is an intuitive and data-efficient way of guiding the robot’s behavior. We propose a modular framework with an automatic embodiment mapping to transfer recorded human hand motions to robotic systems. In this work, we use motion capture to record human motion. We evaluate our approach on eight challenging tasks, in which a robotic hand needs to grasp and manipulate either deformable or small and fragile objects. We test a subset of trajectories in simulation and on a real robot and the overall success rates of about 70% are aligned.


A_Modular_Approach_to_the_Embodiment_of_Hand_Motions_from_Human_Demonstrations.pdf (pdf, 4 MB )

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz