The task of supporting a human operator to understand generated plans, and to explore the plan space, are important problems in automated planning. In this work, we consider the problem of plan explainability and plan space exploration in underwater autonomous vehicle missions. In this context, concepts that are useful for querying the system, such as distance and duration, will not necessarily map directly onto components of the planning model, such as actions. To overcome this difficulty, we focus on an important substructure of these problems: the multi-agent spatial-temporal (MAST) structure. Using this structure, we define a collection of model extensions, which include additional concepts relevant to the MAST structure. We then consider the problem of user-guided plan space exploration, and identify useful query types in this domain, including user queries based on numeric functions. These queries can make use of the extended model, allowing the user to directly reference the new concepts. In an empirical study, we demonstrate the use of the new structure within queries, and compare the new query types in our target domain, and in benchmark domains with the MAST structure. Finally, we report on a qualitative user study, where we investigate the use of these new structural concepts in underwater autonomous vehicle scenarios. Our study indicates that the extended concepts can be used in user queries and agent responses, enabling the user to better communicate their intent in shaping mission objectives, and supporting explanations with more relevant information.

Automated planning requires explicit domain knowledge, typically represented in PDDL, to generate effective solutions. The process of formulating, maintaining, and validating this knowledge is the cornerstone of Knowledge Engineering for Planning and Scheduling (KEPS). Although Large Language Models (LLMs) have shown promise for automated planning tasks, and are gaining popularity in the field, their impact on KEPS remains unexplored. In this paper we investigate the potential of LLMs to streamline and enhance the KEPS field, by taking a close look at the processes used to develop explicit symbolic knowledge models in safety-related applications. The paper's findings are that while LLMs can assist in knowledge acquisition and formulation, human domain expertise and external symbolic validators remain indispensable for ensuring correctness, operationality and completeness of planning applications.

In autonomous vehicle mission planning, supporting human operators to understand and influence the decision-making process is crucial for building the operator’s trust and establishing effective collaboration. However, it has been observed that human and agent representations will typically not align. As a consequence, concepts that are useful for effective human-agent communication, will not necessarily feature in the agent’s representation. Focusing on specific spatial-temporal concepts, we define automatic model extensions, which can introduce these additional concepts. We report on a qualitative user study, where we investigate the use of these new structural concepts in underwater autonomous vehicle scenarios. Our study indicates that the extended concepts can be used in user queries and agent responses, enabling the user to better communicate their intent in shaping mission objectives, and supporting explanations with more relevant information.

The software submission presents a fully working artificial cognition model, which controls a NAO social robot. The model was specifically designed to control a socio-affective companion robot for use in a medical setting. It was deployed using embedded hardware: a Raspberry Pi 4B and a Jetson Nano Board, and an external RGB-D camera. Based on the ROS operating system, this software package includes components for social signal processing, behaviour selection, affective behaviour rendering, and a web-based user interface. The robot's behaviours are selected by a planning system, which generates the robot's behaviours based on the state of the interaction, the progress of the medical procedure, and the user's affective state. The system has been tested in simulated environments and is currently being used in two clinics to perform a usability test and will subsequently be used to carry out a series of clinical trials

Representing and manipulating domain knowledge is essential for developing systems that can visualize plans. This paper presents a novel plan visualisation system called Planning Domain Simulation (PDSim) that employs knowledge representation and manipulation techniques to support the plan visualization process. PDSim can use PDDL or the Unified Planning Library's Python representation as the underlying language for modelling planning problems and provides an interface for users to manipulate this representation through interaction with the Unity game engine and a set of planners. The system’s features include visualising plan components, and their relationships, identifying plan conflicts, and examples applied to real-world problems. The benefits and limitations of PDSim are also discussed, highlighting future research directions in the area.

Affordances describe the possibilities for an agent to perform actions with an object. While the significance of the affordance concept has been previously studied from varied perspectives, such as psychology and cognitive science, these approaches are not always sufficient to enable direct transfer, in the sense of implementations, to artificial intelligence (AI)-based systems and robotics. However, many efforts have been made to pragmatically employ the concept of affordances, as it represents great potential for AI agents to effectively bridge perception to action. In this survey, we review and find common ground amongst different strategies that use the concept of affordances within robotic tasks, and build on these methods to provide guidance for including affordances as a mechanism to improve autonomy. To this end, we outline common design choices for building representations of affordance relations, and their implications on the generalisation capabilities of an agent when facing previously unseen scenarios. Finally, we identify and discuss a range of interesting research directions involving affordances that have the potential to improve the capabilities of an AI agent.

Reasoning about object grasp affordances allows an autonomous agent to estimate the most suitable grasp to execute a task. While current approaches for estimating grasp affordances are effective, their prediction is driven by hypotheses on visual features rather than an indicator of a proposal's suitability for an affordance task. Consequently, these works cannot guarantee any level of performance when executing a task and, in fact, not even ensure successful task completion. In this work, we present a pipeline for self-assessment of grasp affordance transfer (SAGAT) based on prior experiences. We visually detect a grasp affordance region to extract multiple grasp affordance configuration candidates. Using these candidates, we forward simulate the outcome of executing the affordance task to analyse the relation between task outcome and grasp candidates. The relations are ranked by performance success with a heuristic confidence function and used to build a library of affordance task experiences. The library is later queried to perform one-shot transfer estimation of the best grasp configuration on new objects. Experimental evaluation shows that our method exhibits a significant performance improvement up to 11.7% against current state-of-the-art methods on grasp affordance detection. Experiments on a PR2 robotic platform demonstrate our method's highly reliable deployability to deal with real-world task affordance problems.

Planning languages have been used successfully in AI for several decades. Recent trends in AI verification and Explainable AI have raised the question of whether AI planning techniques can be verified. In this paper, we present a novel resource logic, the Proof Carrying Plans (PCP) logic that can be used to verify plans produced by AI planners. The PCP logic takes inspiration from existing resource logics (such as Linear logic and Separation logic) as well as Hoare logic when it comes to modelling states and resource-aware plan execution. It also capitalises on the Curry-Howard approach to logics, in its treatment of plans as functions and plan pre- and post-conditions as types. This paper presents two main results. From the theoretical perspective, we show that the PCP logic is sound relative to the standard possible world semantics used in AI planning. From the practical perspective, we present a complete Agda formalisation of the PCP logic and of its soundness proof. Moreover, we showcase the Curry-Howard, or functional, value of this implementation by supplementing it with the library that parses AI plans into Agda’s proofs automatically. We provide evaluation of this library and the resulting Agda functions. Keywords: AI planning, Verification, Resource Logics, Theorem Proving, Dependent Types.

The soundness and optimality of a plan depends on the correctness of the domain model. Specifying complete domain models can be difficult when interactions between an agent and its environment are complex. We propose a model-based reinforcement learning (MBRL) approach to solve planning problems with unknown models. The model is learned incrementally over episodes using only experiences from the current episode which suits non-stationary environments. We introduce the novel concept of reliability as an intrinsic motivation for MBRL, and a method to learn from failure to prevent repeated instances of similar failures. Our motivation is to improve the learning efficiency and goal-directedness of MBRL. We evaluate our work with experimental results for three planning domains.

For robots to solve real world tasks, they often require the ability to reason about both symbolic and geometric knowledge. We present a framework, called KABouM, for integrating knowledge-level task planning and motion planning in a bounding geometry. By representing symbolic information at the knowledge level, we can model incomplete information, sensing actions and information gain; by representing all geometric entities--objects, robots and swept volumes of motions--by sets of convex polyhedra, we can efficiently plan manipulation actions and raise reasoning about geometric predicates, such as collisions, to the symbolic level. At the geometric level, we take advantage of our bounded convex decomposition and swept volume computation with quadratic convergence, and fast collision detection of convex bodies. We evaluate our approach on a wide set of problems using real robots, including tasks with multiple manipulators, sensing and branched plans, and mobile manipulation.

Cognitive robots, able to adapt their actions based on sensory information and the management of uncertainty, have begun to find their way into manufacturing settings. However, the full potential of these robots has not been fully exploited, largely due to the lack of vertical integration with existing IT infrastructures, such as the manufacturing execution system (MES), as part of a large-scale cyber-physical entity. This paper reports on considerations and findings from the research project STAMINA that is developing such a cognitive cyber-physical system and applying it to a concrete and well-known use case from the automotive industry. Our approach allows manufacturing tasks to be performed without human intervention, even if the available description of the environment-the world model-suffers from large uncertainties. Thus, the robot becomes an integral part of the MES, resulting in a highly flexible overall system.

For robots to solve hard tasks in real-world manufacturing and service contexts, they need to reason about both symbolic and geometric preconditions, and the effects of complex actions. We use an existing Knowledge of Volumes approach to robot task planning (KVP), which facilitates hybrid planning with symbolic actions and continuous-valued robot and object motion, and make two important additions to this approach: (i) new geometric predicates are added for complex object manipulation planning, and (ii) all geometric queries-such as collision and inclusion of objects and swept volumes-are implemented with a single-sided, bounded approximation, which calculates efficient and safe robot motion paths. Our task planning framework is evaluated in multiple scenarios, using concise and generic scenario definitions.

The Workshop on Multimodal, Multi-Party, Real-World Human-Robot Interaction will be held in Istanbul on 16 November 2014, co-located with the 16th International Conference on Multimodal Interaction (ICMI 2014). The workshop objective is to address the challenges that robots face when interacting with humans in real-world scenarios. The workshop brings together researchers from intention and activity recognition, person tracking, robust speech recognition and language processing, multimodal fusion, planning and decision making under uncertainty, and service robot design. The programme consists of two invited talks, three long paper talks, and seven late-breaking abstracts. Information on the workshop and pointers to workshop papers and slides can be found at http://www.macs.hw.ac.uk/~mef3/icmi-2014-workshop-hri/.

Autonomous robots often require domain knowledge to act intelligently in their environment. This is particularly true for robots that use automated planning techniques, which require symbolic representations of the operating environment and the robot's capabilities. However, the task of specifying domain knowledge by hand is tedious and prone to error. As a result, we aim to automate the process of acquiring general common sense knowledge of objects, relations, and actions, by extracting such information from large amounts of natural language text, written by humans for human readers. We present two methods for knowledge acquisition, requiring only limited human input, which focus on the inference of spatial relations from text. Although our approach is applicable to a range of domains and information, we only consider one type of knowledge here, namely object locations in a kitchen environment. As a proof of concept, we test our approach using an automated planner and show how the addition of common sense knowledge can improve the quality of the generated plans.

We address the question of whether service robots that interact with humans in public spaces must express socially appropriate behaviour. To do so, we implemented a robot bartender which is able to take drink orders from humans and serve drinks to them. By using a high-level automated planner, we explore two different robot interaction styles: in the task only setting, the robot simply fulfils its goal of asking customers for drink orders and serving them drinks; in the socially intelligent setting, the robot additionally acts in a manner socially appropriate to the bartender scenario, based on the behaviour of humans observed in natural bar interactions. The results of a user study show that the interactions with the socially intelligent robot were somewhat more efficient, but the two implemented behaviour settings had only a small influence on the subjective ratings. However, there were objective factors that influenced participant ratings: the overall duration of the interaction had a positive influence on the ratings, while the number of system order requests had a negative influence. We also found a cultural difference: German participants gave the system higher pre-test ratings than participants who interacted in English, although the post-test scores were similar.

Robot task planning is an inherently challenging problem, as it covers both continuous-space geometric reasoning about robot motion and perception, as well as purely symbolic knowledge about actions and objects. This paper presents a novel “knowledge of volumes” framework for solving generic robot tasks in partially known environments. In particular, this approach (abbreviated, KVP) combines the power of symbolic, knowledge-level AI planning with the efficient computation of volumes, which serve as an intermediate representation for both robot action and perception. While we demonstrate the effectiveness of our framework in a bimanual robot bartender scenario, our approach is also more generally applicable to tasks in automation and mobile manipulation, involving arbitrary numbers of manipulators.

We introduce a humanoid robot bartender that is capable of dealing with multiple customers in a dynamic, multi-party social setting. The robot system incorporates state-of-the-art components for computer vision, linguistic processing, state management, high-level reasoning, and robot control. In a user evaluation, 31 participants interacted with the bartender in a range of social situations. Most customers successfully obtained a drink from the bartender in all scenarios, and the factors that had the greatest impact on subjective satisfaction were task success and dialogue efficiency.

Natural language generation (NLG) is a major subfield of computational linguistics with a long tradition as an application area of automated planning systems. While current mainstream approaches have largely ignored the planning approach to NLG, several recent publications have sparked a renewed interest in this area. In this article, we investigate the extent to which these new NLG approaches profit from the advances in planner expressiveness and efficiency. Our findings are mixed. While modern planners can readily handle the search problems that arise in our NLG experiments, their overall runtime is often dominated by the grounding step they perform as preprocessing. Furthermore, small changes in the structure of a domain can significantly shift the balance between search and preprocessing. Overall, our experiments show that the off‐the‐shelf planners we tested are unusably slow for nontrivial NLG problem instances. As a result, we offer our domains and experiences as challenges for the planning community.

We investigate the problem of learning action effects in partially observable STRIPS planning domains. Our approach is based on a voted kernel perceptron learning model, where action and state information is encoded in a compact vector representation as input to the learning mechanism, and resulting state changes are produced as output. Our approach relies on deictic features that assume an attentional mechanism that reduces the size of the representation. We evaluate our approach on a number of partially observable planning domains, and show that it can quickly learn the dynamics of such domains, with low average error rates. We show that our approach handles noisy domains, conditional effects, and that it scales independently of the number of objects in a domain.