Publication
Disentangling Interaction Using Maximum Entropy Reinforcement Learning in Multi-Agent Systems
David Rother; Thomas H. Weisswange; Jan Peters
In: Kobi Gal; Ann Nowé; Grzegorz J. Nalepa; Roy Fairstein; Roxana Radulescu (Hrsg.). ECAI 2023 - 26th European Conference on Artificial Intelligence, September 30 - October 4, 2023, Kraków, Poland - Including 12th Conference on Prestigious Applications of Intelligent Systems (PAIS 2023). European Conference on Artificial Intelligence (ECAI), Pages 1994-2001, Frontiers in Artificial Intelligence and Applications, Vol. 372, IOS Press, 2023.
Abstract
Research on multi-agent interaction involving both mul-
tiple artificial agents and humans is still in its infancy. Most recent ap-
proaches have focused on environments with collaboration-focused
human behavior, or providing only a small, defined set of situations.
When deploying robots in human-inhabited environments in the fu-
ture, it will be unlikely that all interactions fit a predefined model of
collaboration, where collaborative behavior is still expected from the
robot. Existing approaches are unlikely to effectively create such be-
haviors in such "coexistence" environments. To tackle this issue, we
introduce a novel framework that decomposes interaction and task-
solving into separate learning problems and blends the resulting poli-
cies at inference time. Policies are learned with maximum entropy re-
inforcement learning, allowing us to create interaction-impact-aware
agents and scale the cost of training agents linearly with the number
of agents and available tasks. We propose a weighting function cov-
ering the alignment of interaction distributions with the original task.
We demonstrate that our framework addresses the scaling problem
while solving a given task and considering collaboration opportuni-
ties in a co-existence particle environment and a new cooking envi-
ronment. Our work introduces a new learning paradigm that opens
the path to more complex multi-robot, multi-human interactions.