Skip to main content Skip to main navigation

Publikation

ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning

Christopher E. Mower; Yuhui Wan; Hongzhan Yu; Antoine Grosnit; Jonas Gonzalez-Billandon; Matthieu Zimmer; Jinlong Wang; Xinyu Zhang; Yao Zhao; Anbang Zhai; Puze Liu; Davide Tateo; Cesar Cadena; Marco Hutter; Jan Peters; Guangjian Tian; Yuzheng Zhuang; Kun Shao; Xingyue Quan; Jianye Hao; Jun Wang; Haitham Bou-Ammar
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2406.19741, Pages 1-26, arXiv, 2024.

Zusammenfassung

We present a framework for intuitive robot programming by non-experts, leveraging natural language prompts and contextual information from the Robot Operating System (ROS). Our system integrates large language models (LLMs), enabling non-experts to articulate task requirements to the system through a chat interface. Key features of the framework include: integration of ROS with an AI agent con- nected to a plethora of open-source and commercial LLMs, automatic extraction of a behavior from the LLM output and execution of ROS actions/services, support for three behavior modes (sequence, behavior tree, state machine), imitation learning for adding new robot actions to the library of possible actions, and LLM reflec- tion via human and environment feedback. Extensive experiments validate the framework, showcasing robustness, scalability, and versatility in diverse scenarios, including long-horizon tasks, tabletop rearrangements, and remote supervisory control. To facilitate the adoption of our framework and support the reproduc- tion of our results, we have made our code open-source. You can access it at: ROS-LLM-Code

Weitere Links