Publication
Composable energy policies for reactive motion generation and reinforcement learning
Julen Urain; Anqi Li; Puze Liu; Carlo D'Eramo; Jan Peters
In: Int. J. Robotics Res. (Hrsg.). International Journal of Robotics Research (IJRR), Vol. 42, No. 10, Pages 827-858, arXiv, 2023.
Abstract
Reactive motion generation problems are usually
solved by computing actions as a sum of policies. However,
these policies are independent of each other and thus, they can
have conflicting behaviors when summing their contributions
together. We introduce Composable Energy Policies (CEP), a
novel framework for modular reactive motion generation. CEP
computes the control action by optimization over the product of
a set of stochastic policies. This product of policies will provide a
high probability to those actions that satisfy all the components
and low probability to the others. Optimizing over the product
of the policies avoids the detrimental effect of conflicting be-
haviors between policies choosing an action that satisfies all the
objectives. Besides, we show that CEP naturally adapts to the
Reinforcement Learning problem allowing us to integrate, in a
hierarchical fashion, any distribution as prior, from multimodal
distributions to non-smooth distributions and learn a new policy
given them. Video in https://sites.google.com/view/composable-
energy-policies/home