We introduce the Block Rearrangement Problem (BRaP), a challenging component of large warehouse management which involves rearranging storage blocks within dense grids to achieve a goal state. We formally define the BRaP as a graph search problem. Building on intuitions from sliding puzzle problems, we propose five search-based solution algorithms, leveraging joint configuration space search, classical planning, multi-agent pathfinding, and expert heuristics. We evaluate the five approaches empirically for plan quality and scalability. Despite the exponential relation between search space size and block number, our methods demonstrate efficiency in creating rearrangement plans for deeply buried blocks in up to 80x80 grids.

Motion planning with simple objectives, such as collision-avoidance and goal-reaching, can be solved efficiently using modern planners. However, the complexity of the allowed tasks for these planners is limited. On the other hand, signal temporal logic (STL) can specify complex requirements, but STL-based motion planning and control algorithms often face scalability issues, especially in large multi-robot systems with complex dynamics. In this paper, we propose an algorithm that leverages the best of the two worlds. We first use a single-robot motion planner to efficiently generate a set of alternative reference paths for each robot. Then coordination requirements are specified using STL, which is defined over the assignment of paths and robots' progress along those paths. We use a Mixed Integer Linear Program (MILP) to compute task assignments and robot progress targets over time such that the STL specification is satisfied. Finally, a local controller is used to track the target progress. Simulations demonstrate that our method can handle tasks with complex constraints and scales to large multi-robot teams and intricate task allocation scenarios.

Multi-agent coordination is crucial for reliable multi-robot navigation in shared spaces such as automated warehouses. In regions of dense robot traffic, local coordination methods may fail to find a deadlock-free solution. In these scenarios, it is appropriate to let a central unit generate a global schedule that decides the passing order of robots. However, the runtime of such centralized coordination methods increases significantly with the problem scale. In this paper, we propose to leverage Graph Neural Network Variational Autoencoders (GNN-VAE) to solve the multi-agent coordination problem at scale faster than through centralized optimization. We formulate the coordination problem as a graph problem and collect ground truth data using a Mixed-Integer Linear Program (MILP) solver. During training, our learning framework encodes good quality solutions of the graph problem into a latent space. At inference time, solution samples are decoded from the sampled latent variables, and the lowest-cost sample is selected for coordination. By construction, our GNN-VAE framework returns solutions that always respect the constraints of the considered coordination problem. Numerical results show that our approach trained on small-scale problems can achieve high-quality solutions even for large-scale problems with 250 robots, being much faster than other baselines.

Multi-robot motion coordination is essential to make robots work safely and efficiently in a shared workspace, ensuring task completion while avoiding interference between each other’s motions. Achieving online replanning and scaling to large numbers of robots are particularly challenging. In this paper, we present a motion coordination method for multiple robot manipulators with given targets. The approach separates the problem into a global coordination stage and a local trajectory replanning stage. Online reactivity and scalability are achieved by means of coordination in a reduced space and efficient trajectory replanning. We first introduce the method for systems with two robots, then extend it to systems with an arbitrary number of robots. The method determines whether kinematically-feasible and non-interfering trajectories leading all the robots to their targets exist. The approach generates time-efficient trajectories if solutions exist, or provides information for switching targets if solutions cannot be found. We show formally and empirically that the method has low computational overhead and scales quadratically with the number of robots. Experiments are conducted with up to three real 7-DOF robots and up to ten simulated robots.Note to Practitioners—In underground mining, a key process is that of tunneling, i.e., drilling and blasting rock to excavate tunnels that lead to sources of ore. Drilling is carried out by rigs equipped with multiple robotic arms. A key factor affecting the efficiency of tunneling is the time to completion of drilling operations. In current industrial practice, these operations are carried out manually by an operator steering the arms on the drill rig. Time to completion can be drastically reduced if the arms could operate concurrently, intelligently optimizing and coordinating their motions. However, existing methods for multi-arm motion coordination are inadequate, as they fail to cater to one or more of the following real-world constraints: several robot arms work in close proximity and their workspaces are overlapping; task completion times are uncertain due to rock density and drill bit breakage; and contingencies such as unexpected delays in motion or stops sometimes happen. These constraints exist also in other industrial applications, like manufacturing and assembly. This paper proposes a framework which enables multiple high-DOF robot manipulators to safely and efficiently work in a shared workspace. The method allows to adjust robot trajectories while robots move, and is shown to scale well with the number of robots. Motions are generated and adjusted by time-optimal trajectory planning, whose computational overhead is small enough for online operation and benefits task completion efficiency, safety, and productivity. Experiments using three 7-DOF robots suggest that this approach is practically feasible, and simulations with up to 10 robots attest to its scalability.

This paper deals with Multi-robot Trajectory Planning, that is, the problem of computing trajectories for multiple robots navigating in a shared space while minimizing for control energy. Approaches based on trajectory optimization can solve this problem optimally. However, such methods are hampered by complex robot dynamics and collision constraints that couple robot's decision variables. We propose a distributed multi-robot optimization algorithm (DiMOpt) that addresses these issues by exploiting (1) consensus optimization strategies to tackle coupling collision constraints, and (2) a single-robot sequential convex programming method for efficiently handling non-convexities introduced by dynamics. We compare DiMOpt with a baseline centralized multi-robot sequential convex programming algorithm (SCP). We empirically demonstrate that DiMOpt scales well for large fleets of robots while computing solutions faster and with lower costs. Finally, DiMOpt is an iterative algorithm that finds feasible trajectories before converging to a locally optimal solution, and results suggest the quality of such fast initial solutions is comparable to a converged solution computed via SCP.

This paper addresses the problem of automating fleets of robots with non-holonomic dynamics. Previously studied methods either specialize in facets of this problem, that is, one or a combination of multi-robot goal allocation, motion planning, and coordination, and typically acrifice optimality and completeness for scalability. We propose an approach that constructs an abstract multi-robot roadmap in a reduced configuration space, where we account for environment connectivity and interference cost between robots occupying the same polygons. Querying the road-map results in a robot-goal assignment and abstract multi-robot trajectory. This is then exploited to de-compose the original problem into smaller problems, each of which is solved with a multi-robot motion planner that accounts for kinodynamic constraints. We validate the approach experimentally to demonstrate the advantage of considering task assignment and motion planning holistically, and explore some methods for balancing solution quality and computational efficiency.

A standing challenge in multirobot systems is to realize safe and efficient motion planning and coordination methods that are capable of accounting for uncertainties and contingencies. The challenge is rendered harder by the fact that robots may be heterogeneous and that their plans may be posted asynchronously. Most existing approaches require constraints on the infrastructure or unrealistic assumptions on robot models. In this article, we propose a centralized, loosely-coupled supervisory controller that overcomes these limitations. The approach responds to newly posed constraints and uncertainties during trajectory execution, ensuring at all times that planned robot trajectories remain kinodynamically feasible, that the fleet is in a safe state, and that there are no deadlocks or livelocks. This is achieved without the need for hand-coded rules, fixed robot priorities, or environment modification. We formally state all relevant properties of robot behavior in the most general terms possible, without assuming particular robot models or environments, and provide both formal and empirical proof that the proposed fleet control algorithms guarantee safety and liveness.

Future robots should follow human social norms to be useful and accepted in human society. In this paper, we show how prior knowledge about social norms, represented using an existing normative framework, can be used to (1) guide reinforcement learning agents towards normative policies, and (2) re-use (transfer) learned policies in novel domains. The proposed method is not dependent on a particular reinforcement learning algorithm and can be seen as a means to learn abstract procedural knowledge based on declarative domain-independent semantic specifications.

The video describes the novel concept of “culturally competent robotics”, which is the main focus of the project CARESSES (Culturally-Aware Robots and Environmental Sensor Systems for Elderly Support). CARESSES a multidisciplinary project whose goal is to design the first socially assistive robots that can adapt to the culture of the older people they are taking care of. Socially assistive robots are required to help the users in many ways including reminding them to take their medication, encouraging them to keep active, helping them keep in touch with family and friends. The video describes a new generation of robots that will perform their actions with attention to the older person's customs, cultural practices and individual preferences.

We present a novel modelling and planning approach for multi-robot systems under uncertain travel times. The approach uses generalised stochastic Petri nets (GSPNs) to model desired team behaviour, and allows to specify safety constraints and rewards. The GSPN is interpreted as a Markov decision process (MDP) for which we can generate policies that optimise the requirements. This representation is more compact than the equivalent multi-agent MDP, allowing us to scale better. Furthermore, it naturally allows for asynchronous execution of the generated policies across the robots, yielding smoother team behaviour. We also describe how the integration of the GSPN with a lower-level team controller allows for accurate expectations on team performance. We evaluate our approach on an industrial scenario, showing that it outperforms hand-crafted policies used in current practice.

The way in which humans behave, speak and interact is deeply influenced by their culture. For example, greeting is done differently in France, in Sweden or in Japan; and the average interpersonal distance changes from one cultural group to the other. In order to successfully coexist with humans, robots should also adapt their behavior to the culture, customs and manners of the persons they interact with. In this paper, we deal with an important ingredient of cultural adaptation: how to generate robot plans that respect given cultural preferences, and how to execute them in a way that is sensitive to those preferences. We present initial results in this direction in the context of the CARESSES project, a joint EU-Japan effort to build culturally competent assistive robots.

This paper is concerned with automating fleets of autonomous robots. This involves solving a multitude of problems, including goal assignment, motion planning, and coordination, while maximizing some performance criterion. While methods for solving these sub-problems have been studied, they address only a facet of the overall problem, and make strong assumptions on the use-case, on the environment, or on the robots in the fleet. In this paper, we formulate the overall fleet management problem in terms of Optimal Control. We describe a scheme for solving this problem in the particular case of fleets of non-holonomic robots navigating in an environment with obstacles. The method is based on a two-phase approach, whereby the first phase solves for fleet-wide boolean decision variables via Mixed Integer Quadratic Programming, and the second phase solves for real-valued variables to obtain an optimized set of trajectories for the fleet. Examples showcasing the features of the method are illustrated, and the method is validated experimentally.

Social norms are the understandings that govern the behavior of members of a society. As such, they regulate communication, cooperation and other social interactions. Robots capable of reasoning about social norms are more likely to be recognized as an extension of our human society. However, norms stated in a form of the human language are inherently vague and abstract. This allows for applying norms in a variety of situations, but if the robots are to adhere to social norms, they must be capable of translating abstract norms to the robotic language. In this paper we use a notion of institution to realize social norms in real robotic systems. We illustrate our approach in a case study, where we translate abstract norms into concrete constraints on cooperative behaviors of humans and robots. We investigate the feasibility of our approach and quantitatively evaluate the performance of our framework in 30 real experiments with user-based evaluation with 40 participants.

Cultural adaptation, i.e., the matching of a robot's behaviours to the cultural norms and preferences of its user, is a well known key requirement for the success of any assistive application. However, culture-dependent robot behaviours are often implicitly set by designers, thus not allowing for an easy and automatic adaptation to different cultures. This paper presents a method for the design of culture-aware robots, that can automatically adapt their behaviour to conform to a given culture. We propose a mapping from cultural factors to related parameters of robot behaviours which relies on linguistic variables to encode heterogeneous cultural factors in a uniform formalism, and on fuzzy rules to encode qualitative relations among multiple variables. We illustrate the approach in two practical case studies.

We introduce a variant of the multi-vehicle routing problem which accounts for nonholonomic constraints and dense, dynamic obstacles, called MVRP-DDO. The problem is strongly motivated by an industrial mining application. This paper illustrates how MVRP-DDO relates to other extensions of the vehicle routing problem. We provide an application-independent formulation of MVRP-DDO, as well as a concrete instantiation in a surface mining application. We propose a multi-abstraction search approach to compute an executable plan for the drilling operations of several machines in a very constrained environment. The approach is evaluated in terms of makespan and computation time, both of which are hard industrial requirements.

Cultural competence is a well known requirement for an effective healthcare, widely investigated in the nursing literature. We claim that personal assistive robots should likewise be culturally competent, aware of general cultural characteristics and of the different forms they take in different individuals, and sensitive to cultural differences while perceiving, reasoning, and acting. Drawing inspiration from existing guidelines for culturally competent healthcare and the state-of-the-art in culturally competent robotics, we identify the key robot capabilities which enable culturally competent behaviours and discuss methodologies for their development and evaluation.

Proactive cognitive agents need to be capable of both generating their own goals and enacting them. In this paper, we cast this problem as that of maintaining equilibrium, that is, seeking opportunities to act that keep the system in desirable states while avoiding undesirable ones. We characterize desirability of states as graded preferences, using mechanisms from the field of fuzzy logic. As a result, opportunities for an agent to act can also be graded, and their relative preference can be used to infer when and how to act. This paper provides a formal description of our computational framework, and illustrates how the use of degrees of desirability leads to well-informed choices of action.

Reliable autonomous navigation is still a challenging problem for robots with simple and inexpensive hardware. A key difficulty is the need to maintain an internal map of the environment and an accurate estimate of the robot's position in this map. Recently, a stigmergic approach has been proposed in which a navigation map is stored into the environment, on a grid of RFID tags, and robots use it to optimally reach predefined goal points without the need for internal maps. While effective, this approach is limited to a predefined set of goal points. In this paper, we extend this approach to enable robots to travel to any point on the RFID floor, even if it was not previously identified as a goal location, as well as to keep a safe distance from any given critical location. Our approach produces safe, repeatable and quasi-optimal trajectories without the use of internal maps, self localization, or path planning. We report experiments run in a real apartment equipped with an RFID floor, in which a service robot either reaches or avoids a user who wears slippers equipped with an RFID tag reader.

An important contribution of AI to Robotics is the model-centred approach, whereby competent robot behaviour stems from automated reasoning in models of the world which can be changed to suit different environments, physical capabilities and tasks. However models need to capture diverse (and often application-dependent) aspects of the robot’s environment and capabilities. They must also have good computational properties, as robots need to reason while they act in response to perceived context. In this article, we investigate the use of a meta-CSP-based technique to interleave reasoning in diverse knowledge types. We reify the approach through a robotic waiter case study, for which a particular selection of spatial, temporal, resource and action KR formalisms is made. Using this case study, we discuss general principles pertaining to the selection of appropriate KR formalisms and jointly reasoning about them. The resulting integration is evaluated both formally and experimentally on real and simulated robotic platforms.

Planning for robots in environments co-inhabited by humans entails handling exogenous events during plan execution. Such events require plans to be continuously adapted to ensure that they remain "human-aware", i.e., adherent to human preferences and needs. We use an approach whereby human-awareness is enforced through so-called interaction constraints. Interaction constraints are used to infer context and appropriate goals online. The current plan is modified at run time so as to achieve courses of action that are continuously human-aware. The approach is evaluated in a research facility environment in which we simulate multiple days of planning and execution.

Real-world robotic systems have to perform reliably in uncertain and dynamic environments. State-of-the-art cognitive robotic systems use an abstract symbolic representation of the real world for high-level reasoning. Some aspects of the world, such as object dynamics, are inherently difficult to capture in an abstract symbolic form, yet they influence whether the executed action will succeed or fail. This paper presents an integrated system that uses a physics-based simulation to predict robot action results and durations, combined with a Hierarchical Task Network (HTN) planner and semantic execution monitoring. We describe a fully integrated system in which a Semantic Execution Monitor (SEM) uses information from the planning domain to perform functional imagination. Based on information obtained from functional imagination, the robot control system decides whether it is necessary to adapt the plan currently being executed. As a proof of concept, we demonstrate a PR2 able to carry tall objects on a tray without the objects toppling. Our approach achieves this by simulating robot and object dynamics. A validation shows that robot action results in simulation can be transferred to the real world. The system improves on state-of-the-art AI plan-based systems by feeding simulated prediction results back into the execution system.

Plan-based robot control has to consider a multitude of aspects of tasks at once, such as task dependency, time, space, and resource usage. Hybrid planning is a strategy for treating them jointly. However, by incorporating all these aspects into a hybrid planner, its search space is huge by construction. This paper introduces the planner CHIMP, which is based on meta-CSP planning to represent the hybrid plan space and uses hierarchical planning as the strategy for cutting efficiently through this space. The paper makes two contributions: First, it describes how HTN planning is integrated into meta-CSP reasoning leading to a planner that can reason about different forms of knowledge and that is fast enough to be used on a robot. Second, it demonstrates CHIMP's task merging capabilities, i.e., the unification of different tasks from different plan parts, resulting in plans that are more efficient to execute. It also allows to merge new tasks online into a plan that is being executed. This is demonstrated on a PR2 robot.

Robots for the elderly are a particular category of home assistive robots, helping people in the execution of daily life tasks to extend their independent life. Such robots should be able to determine the level of independence of the user and track its evolution over time, to adapt the assistance to the person capabilities and needs. Human Activity Recognition systems employ various sensing strategies, relying on environmental or wearable sensors, to recognize the daily life activities which provide insights on the health status of a person. The main contribution of the article is the design of an heterogeneous information management framework, allowing for the description of a wide variety of human activities in terms of multi-modal environmental and wearable sensing data and providing accurate knowledge about the user activity to any assistive robot.

Consider a family whose home is equipped with several service robots. The actions planned for the robots must adhere to Interaction Constraints (ICs) relating them to human activities and preferences. These constraints must be sufficiently expressive to model both temporal and logical dependencies among robot actions and human behavior, and must accommodate incomplete information regarding human activities. In this paper we introduce an approach for automatically generating plans that are conformant wrt. given ICs and partially specified human activities. The approach allows to separate causal reasoning about actions from reasoning about ICs, and we illustrate the computational advantage this brings with experiments on a large-scale (semi-)realistic household domain with hundreds of human activities and several robots.

AI-based solutions for robot planning have so far focused on very high-level abstractions of robot capabilities and of the environment in which they operate. However, to be useful in a robotic context, the model provided to an AI planner should afford both symbolic and metric constructs; its expressiveness should not hinder computational efficiency; and it should include causal, spatial, temporal and resource aspects of the domain. We propose a planner grounded on well-founded constraint-based calculi that adhere to these requirements. A proof of completeness is provided, and the flexibility and portability of the approach is validated through several experiments on real and simulated robot platforms.

Early detection and adaptive support to changing individual needs related to ageing is an important challenge in today's society. In this paper we present a system called GiraffPlus that aims at addressing such a challenge and is developed in an on-going European project. The system consists of a network of home sensors that can be automatically configured to collect data for a range of monitoring services; a semi-autonomous telepresence robot; a sophisticated context recognition system that can give high-level and long term interpretations of the collected data and respond to certain events; and personalized services delivered through adaptive user interfaces for primary users. The system performs a range of services including data collection and analysis of long term trends in behaviors and physiological parameters (e.g. relating to sleep or daily activity); warnings, alarms and reminders; and social interaction through the telepresence robot. The latter is based on the Giraff telepresence robot, which is already in place in a number of homes. A distinctive aspect of the project is that the GiraffPlus system will be installed and evaluated in at least 15 homes of elderly people. This paper provides a general overview of the GiraffPlus system and its evaluation.

Unexpected contingencies in robot execution may induce a cascade of effects, especially when multiple robots are involved. In order to effectively adapt to this, robots need the ability to reason along multiple dimensions at execution time. We propose an approach to closed-loop planning capable of generating configuration plans, i.e., action plans for multirobot systems which specify the causal, temporal, resource and information dependencies between individual sensing, computation, and actuation components. The key feature which enables closed loop performance is that configuration plans are represented as constraint networks, which are shared between the planner and the executor and are continuously updated during execution. We report experiments run both in simulation and on real robots, in which a fault in one robot is compensated through different types of plan modifications at run time.

Coordinating multiple autonomous ground vehicles is paramount to many industrial applications. Vehicle trajectories must take into account temporal and spatial requirements, e.g., usage of floor space and deadlines on task execution. In this paper we present an approach to obtain sets of alternative execution patterns (called trajectory envelopes) which satisfy these requirements and are conflict-free. The approach consists of multiple constraint solvers which progressively refine trajectory envelopes according to mission requirements. The approach leverages the notion of least commitment to obtain easily revisable trajectories for execution.

This article describes aspects of a fully implemented artificial intelligence (AI) system that integrates multiple intelligent components to actively assist an elderly person at home. Specifically, we describe how constraint‐based scheduling technology is used to actively monitor a pattern of activities executed by the person and how detected temporal constraint violations are used to trigger meaningful and contextualized proactive interactions. This article also presents a psychological evaluation of the system focusing on elderly people's attitudes, in which system acceptability, perceived utility, interaction modality, and emotional response are considered.

Intelligent environments are increasingly rich in ubiquitous sensing capabilities that can be leveraged to know which actions a user is engaged in at any given moment in time. The ability of an intelligent environment to recognize a high-level plan of activities performed by the user in a smart home would allow to construct proactive services, such as reminding, forecasting and providing timely physical support. This article proposes an approach to human activity recognition based on temporal planning. The approach leverages on one hand the ubiquitous sensors provided by the PEIS-Home, a sensor-rich intelligent environment, and, on the other hand, the temporal representation and reasoning capabilities of OMPS, a constraint-based temporal planning and scheduling framework.

The aim of this article is to bring forth the issue of integrating the services provided by intelligent artifacts in Ambient Intelligence applications. Specifically, we propose a Distributed Constraint Optimization procedure for achieving a functional integration of intelligent artifacts in a smart home. To this end, we employAdopt‐N, a state‐of‐the‐art algorithm for solving Distributed Constraint Optimization Problems (DCOP). This article attempts to state the smart home coordination problem in general terms, and provides the details of a DCOP‐based approach by describing a case study taken from theRoboCareproject. More specifically, we show how (1) DCOP is a convenient metaphor for casting smart home coordination problems, and (2) the specific features which distinguishAdopt‐Nfrom other algorithms for DCOP represent a strong asset in the smart home domain.