Publikation
ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning
Christopher E. Mower; Yuhui Wan; Hongzhan Yu; Antoine Grosnit; Jonas Gonzalez-Billandon; Matthieu Zimmer; Jinlong Wang; Xinyu Zhang; Yao Zhao; Anbang Zhai; Puze Liu; Davide Tateo; Cesar Cadena; Marco Hutter; Jan Peters; Guangjian Tian; Yuzheng Zhuang; Kun Shao; Xingyue Quan; Jianye Hao; Jun Wang; Haitham Bou-Ammar
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2406.19741, Pages 1-26, arXiv, 2024.
Zusammenfassung
We present a framework for intuitive robot programming by non-experts, leveraging
natural language prompts and contextual information from the Robot Operating
System (ROS). Our system integrates large language models (LLMs), enabling
non-experts to articulate task requirements to the system through a chat interface.
Key features of the framework include: integration of ROS with an AI agent con-
nected to a plethora of open-source and commercial LLMs, automatic extraction of
a behavior from the LLM output and execution of ROS actions/services, support for
three behavior modes (sequence, behavior tree, state machine), imitation learning
for adding new robot actions to the library of possible actions, and LLM reflec-
tion via human and environment feedback. Extensive experiments validate the
framework, showcasing robustness, scalability, and versatility in diverse scenarios,
including long-horizon tasks, tabletop rearrangements, and remote supervisory
control. To facilitate the adoption of our framework and support the reproduc-
tion of our results, we have made our code open-source. You can access it at:
ROS-LLM-Code
