Kenneweg S, Deigmöller J, Cimiano P, Eggert J. Benchmarking the Ability of Large Language Models to Reason About Event Sets. In: Proceedings of the 16th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management. SCITEPRESS - Science and Technology Publications; 2024: 74-82.

In the pursuit of fully autonomous robotic systems capable of taking over tasks traditionally performed by humans, the complexity of open-world environments poses a considerable challenge. Addressing this imperative, this study contributes to the field of Large Language Models (LLMs) applied to task and motion planning for robots. We propose a system architecture that orchestrates a seamless interplay between multiple cognitive levels, encompassing reasoning, planning, and motion generation. At its core lies a novel replanning strategy that handles physically grounded, logical, and semantic errors in the generated plans. We demonstrate the efficacy of the proposed feedback architecture, particularly its impact on executability, correctness, and time complexity via empirical evaluation in the context of a simulation and two intricate real-world scenarios: blocks world, barman and pizza preparation.

How can a robot provide unobtrusive physical support within a group of humans? We present Attentive Support, a novel interaction concept for robots to support a group of humans. It combines scene perception, dialogue acquisition, situation understanding, and behavior generation with the common-sense reasoning capabilities of Large Language Models (LLMs). In addition to following user instructions, Attentive Support is capable of deciding when and how to support the humans, and when to remain silent to not disturb the group. With a diverse set of scenarios, we show and evaluate the robot’s attentive behavior, which supports and helps the humans when required, while not disturbing if no help is needed.

Manipulation-relevantcommon-sense knowledge is crucial to support actionplanning for complex tasks.In particular, instrumentality information of what can be done with certain tools can be used to limit the search space which is growing exponentially with the number of viable options.Typical sources for such knowledge, structured common-sense knowledge bases such as ConceptNet or WebChild, provide a limited amount of information which also varies drastically across different domains.Considering the recent success of pre-trained language models such as BERT, we investigate whether common-sense information can directly be extracted from semi-structured text with an acceptable annotation effort.Concretely, we compare the common-sense relations obtained from ConceptNet versus those extracted with BERT from large recipe databases.In this context, we propose a scoring function, based on the WordNet taxonomy to match specific terms to more general ones, enabling a rich evaluation against a set of ground-truth relations.

In this paper, we introduce Memory Nets, a knowledge representation targeted at Autonomous Intelligent Agents (IAs) operating in real world. The main focus is on a knowledge base (KB) that on the one hand is able to leverage the large body of openly available semantic information, and on the other hand allows to incrementally accumulate additional knowledge from situated interaction. Such a KB can only rely on operable semantics fully contained in the knowledge base itself, avoiding any type of hidden semantics in the KB attributes, such as human-interpretable identifier. In addition, it has to provide means for tightly coupling the internal representation to real-world events. We propose a KB structure and inference processes based on a knowledge graph that has a small number of link types with operational semantics only, and where the main information lies in the complex patterns and connectivity structures that can be build incrementally using these links. We describe the basic domain independent features of Memory Nets and the relation to measurements and actuator capabilities as available by autonomous entities, with the target of providing a KB framework for researching how to create IAs that continuously expand their knowledge about the world.