Learning abstract state representations and knowledge is crucial for long-horizon robot planning. We present InterPreT, an LLM-powered framework for robots to learn symbolic predicates from language feedback of human non-experts during embodied interaction. The learned predicates provide relational abstractions of the environment state, facilitating the learning of symbolic operators that capture action preconditions and effects. By compiling the learned predicates and operators into a PDDL domain file on-the-fly, InterPreT allows effective planning toward arbitrary in-domain goals using a PDDL planner.
In both simulated and real-world robot manipulation domains, we demonstrate that InterPreT reliably uncovers the key predicates and operators governing the environment dynamics. Although learned from simple training tasks, these predicates and operators exhibit strong generalization to novel tasks with significantly higher complexity. In the most challenging generalization setting, InterPreT attains success rates of 73% in simulation and 40% in the real world, substantially outperforming baseline methods.
InterPreT leverages GPT-4 to generate and iteratively refine predicate functions (in Python) based on language feedback. Then a cluster-then-search algorithm learns symbolic operators from collected interaction data. At test time, we prompt GPT-4 to translate language goal specifications into symbolic goals for to PDDL planning with the learned PDDL domain.
InterPreT explicitly learns grounded predicates and opeators during embodied interation. The learned predicates and operators reliably capture regularities of the environment, allowing effective planning for novel tasks that are more complex than the training ones and require long-horizon planning.
On the contrary, LLM-based planners rely on ungrounded world knowledge for planning. Therefore, they fall short in generalizing to tasks more complex than those in the few-shot examples.
@inproceedings{han2024interpret,
title={InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planning},
author={Muzhi Han and Yifeng Zhu and Song-Chun Zhu and Ying Nian Wu and Yuke Zhu},
booktitle = {Robotics: Science and Systems (RSS)},
year = {2024}
}