We propose an incremental, modular, and extensible method for learning robotic manipulation tasks using a limited number of demonstrations provided in Virtual Reality, while assuming minimal prior information about the objects to be manipulated. The developed framework enables an incremental training process in which the operator first demonstrates specialized tasks to the robotic system, and subsequently more complex tasks, exploiting the skills learned during the previous phases. We illustrate and discuss the method at work considering picking tasks performed by manipulators equipped with multi-fingered sensorized hands. The experimental evaluation highlights the feasibility and advantage of the proposed method, particularly in terms of modularity, low number of demonstrations, and reliability of the trained system.

Sanitizing railway stations is a relevant issue, primarily due to the recent evolution of the Covid-19 pandemic. In this work, we propose a multi-robot approach to sanitize railway stations based on a distributed Deep Q-Learning technique. The proposed framework relies on anonymous data from existing WiFi networks to dynamically estimate crowded areas within the station and to develop a heatmap of prioritized areas to be sanitized. Such heatmap is then provided to a team of cleaning robots - each endowed with a robot-specific convolutional neural network - that learn how to effectively cooperate and sanitize the station's areas according to the associated priorities. The proposed approach is evaluated in a realistic simulation scenario provided by the Italian largest railways station: Roma Termini. In this setting, we consider different case studies to assess how the approach scales with the number of robots and how the trained system performs with a real dataset retrieved from a one-day data recording of the station's WiFi network.

Bayesian Preference Elicitation (PE) is an active learning technique aimed at discovering users’ preferences through a suitable sequence of queries. One of its prominent applications consists in providing personalized product recommendations in e-Commerce scenarios. In these settings, where catalogs may contain hundreds or thousands of different products, it becomes crucial to find a good trade-off between recommendation quality and computational cost.In this paper we introduce QUEST, a highly configurable PE framework supporting arbitrary value functions on multi-attribute product domains. We use QUEST to compare established and novel query selection methodologies to achieve real-time user interaction, while preserving recommendation quality. In particular, the experimental assessment shows that a novel uncertainty-based query selection strategy outperforms the largely used value-of-information (VOI) both in accuracy and execution time.

In critical infrastructures like airports, much care has to be devoted in protecting radio communication networks from external electromagnetic interference. Protection of such mission-critical radio communication networks is usually tackled by exploiting radiogoniometers: at least three suitably deployed radiogoniometers, and a gateway gathering information from them, permit to monitor and localise sources of electromagnetic emissions that are not supposed to be present in the monitored area. Typically, radiogoniometers are connected to the gateway through relay nodes. As a result, some degree of fault-tolerance for the network of relay nodes is essential in order to offer a reliable monitoring. On the other hand, deployment of relay nodes is typically quite expensive. As a result, we have two conflicting requirements: minimise costs while guaranteeing a given fault-tolerance. In this paper, we address the problem of computing a deployment for relay nodes that minimises the overall cost while at the same time guaranteeing proper working of the network even when some of the relay nodes (up to a given maximum number) become faulty (fault-tolerance). We show that, by means of a computation-intensive pre-processing on a HPC infrastructure, the above optimisation problem can be encoded as a 0/1 Linear Program, becoming suitable to be approached with standard Artificial Intelligence reasoners like MILP, PB-SAT, and SMT/OMT solvers. Our problem formulation enables us to present experimental results comparing the performance of these three solving technologies on a real case study of a relay node network deployment in areas of the Leonardo da Vinci Airport in Rome, Italy.

Cobots (Collaborative robots) are increasingly employed in industrial workplaces, assisting human operators in decreasing the weight and the repetitiveness of their activities. In order to promote the acceptance of cobots, novel interaction paradigms enabling intuitive collaboration between humans and robots are needed. In this work, we propose a variable admittance control framework based on virtual fixtures, in which the damping and the stiffness of the admittance controller are on-line adjusted to increase the effectiveness of the co-manipulation task. Virtual paths are generated to support and guide the operator during collaborative task execution, providing him/her an over-responsive compliant system when the robot is far from the targets and a precise heavy tool in their proximity. The proposed approach has been compared with a fixed admittance controller in an industrial use case consisting of a human operator interacting with a Kuka LBR IIWA arm. The collected results demonstrate the effectiveness of the proposed approach.

We present a novel multimodal interaction framework supporting robust human-robot communication. We consider a scenario where a human operator can exploit multiple communication channels to interact with one or more robots in order to accomplish shared tasks. Moreover, we assume that the human is not fully dedicated to the robot control, but also involved in other activities, hence only able to interact with the robotic system in a sparse and incomplete manner. In this context, several human or environmental factors could cause errors, noise and wrong interpretations of the commands. The main goal of this work is to improve the robustness of human-robot interaction systems in similar situations. In particular, we propose a multimodal fusion method based on the following steps: for each communication channel, unimodal classifiers are firstly deployed in order to generate unimodal interpretations of the human inputs; the unimodal outcomes are then grouped into different multimodal recognition lines, each representing a possible interpretation of a sequence of multimodal inputs; these lines are finally assessed in order to recognize the human commands. We discuss the system at work in a case study in which a human rescuer interacts with a team of flying robots during Search & Rescue missions. In this scenario, we present and discuss real world experiments to demonstrate the effectiveness of the proposed framework.

We present a system suitable for human multi-robot interaction that supports the operator in the robot selection process. The proposed framework allows a human to issue commands to a robotic team without an explicit robot selection in so enabling a fluent interaction. This work is framed in the operative context of the SHERPA project [1], which proposes the deployment of a robotic platform for Search & Rescue in an alpine scenario and assumes the presence of a human rescuer that can orchestrate the robots operations with multimodal commands. In this context, implicit robot selection is mainly motivated by fast communication and the difficulties to distinguish different robots of similar shape in a hazardous environment and in adverse weather conditions. In the proposed approach, each robot of the team can evaluate the probability to be referred in an incomplete command, considering its actual capabilities along with geometrical and contextual information. We describe the overall system architecture focusing on the human intention recognition process. The proposed framework is trained and evaluated in a simulated case study.

A human-robot interaction system should be capable of adapting the execution of cooperative plans with respect to complex human activities and interventions. In this paper, we present an integrated framework that exploits attentional supervision and contention scheduling to combine human-aware planning, plan execution, and natural human-robot interaction. Specifically, in the proposed approach, hierarchical cooperative plans are exploited as top-down attentional guidance for the robotic executive system, which can flexibly orchestrate the task activities while reacting to environmental stimuli and human behaviors. We describe the overall framework discussing some case studies in human-robot collaborative scenarios.

We present a multimodal attentional interface suitable for a human operator that monitors and controls the activities of a team of drones during search and rescue missions. We consider a scenario where the operator is a component of the rescue team, hence not fully dedicated to the robots, but only able to interact with them with sparse and incomplete commands. In this context, an adaptive interface is needed to support the user situation awareness and to enable an effective interaction with the drones. In this work, we propose a multimodal attention-based interface designed for this domain. This framework is to filter the information flow towards the operator selecting and adapting the communication mode according to the context and the human state. We illustrate the features of the adaptive system along with an initial assessment in a simulated scenario.

Flexible execution of structured tasks is a relevant issue in cognitive robotics and human-robot interaction. In this work, we address this problem presenting a framework that integrates planning and attentional regulations for flexible and interactive execution of human-robot cooperative tasks. In the proposed approach, attentional top-down and bottom-up mechanisms are deployed to guide the execution of generated hierarchical plans while managing conflicts and decisional impasses. We provide an overview of the proposed framework discussing the system at work in different scenarios. In particular, we focus on flexible execution of multiple plans and interactive plan execution guided by attentional manipulation.

We propose a framework where the human-robot interaction is modeled as a multimodal dialogue which is regulated by an attentional system that guides the system towards the execution of structured tasks. We introduce a simple case study to illustrate the system at work in different conditions considering top-down regulations and dialogue flows in synergic and conflicting situations.

In this work, we present a reliable and continuous gesture recognition method that supports a natural and flexible interaction between the human and the robot. The aim is to provide a system that can be trained online with few samples and can cope with intra user variability during the gesture execution. The proposed approach relies on the generation of an ad-hoc Hidden Markov Model (HMM) for each gesture exploiting a direct estimation of the parameters. Each model represents the best prototype candidate from the associated gesture training set. The generated models are then employed within a continuous recognition process that provides the probability of each gesture at each step. The proposed method is evaluated in two case studies: a hand-performed letters recognizer and a natural gesture recognizer. Finally, we show the overall system at work in a simple human-robot interaction scenario.

We present an approach to mixed initiative control for unmanned aerial vehicles (UAVs) where sliding autonomy is supported by mixed-initiative planning and haptic feedback. In the proposed framework, we assume that an autonomous system can plan and execute robotic tasks while a human operator can provide interventions when necessary receiving a force feedback. The haptic feedback is associated with the sensation about how the system is diverging from the planned operations. We tested the system at work in virtual and real environments considering simple navigation tasks. We compared the performance of human operators with or without the assistance of the force feedback. The collected results support the hypothesis that the proposed approach enables effective and intuitive mixed-initiative control.

We propose a framework where the human-robot interaction is modeled as a multimodal dialogue which is regulated by an attentional system that guides the robot towards the execution of structured tasks. Specifically, we propose an approach where the dialogue between the human and the robot is represented as a Partially Obervable Markov Decision Process (POMDP), while the associated dialogue policy is enhanced by top-down attentional mechanisms that provide contextual and task-related contents. We introduce simple case studies that illustrate the system at work in different conditions considering top-down regulations and dialogue flows in synergistic and conflicting situations.

This paper presents a POMDP-based dialogue system for multimodal human-robot interaction (HRI). Our aim is to exploit a dialogical paradigm to allow a natural and robust interaction between the human and the robot. The proposed dialogue system should improve the robustness and the flexibility of the overall interactive system, including multimodal fusion, interpretation, and decision-making. The dialogue is represented as a Partially Observable Markov Decision Process (POMDPs) to cast the inherent communication ambiguity and noise into the dialogue model. POMDPs have been used in spoken dialogue systems, mainly for tourist information services, but their application to multimodal human-robot interaction is novel. This paper presents the proposed model for dialogue representation and the methodology used to compute a dialogue strategy. The whole architecture has been integrated on a mobile robot platform and has bee n tested in a human-robot interaction scenario to assess the overall performances with respect to baseline controllers.

Safety critical planning and execution is a crucial issue in autonomous systems. This paper proposes a methodology for controller synthesis suitable for timeline-based planning and demonstrates its effectiveness in a space domain where robustness of execution is a crucial property. The proposed approach uses Timed Game Automata (TGA) for formal modeling and the UPPAAL-TIGA model checker for controllers synthesis. An experimental evaluation is performed using a real-world control system.

Human safety and effective human-robot communication are main concerns in HRI applications. In order to achieve such goals, a system should be very robust, allowing little chance for misunderstanding the user's commands. Moreover, the system should permit natural interaction reducing the time and the effort needed to achieve tasks. The main purpose of this work is to develop a general framework for flexible and multimodal human-robot communication. The proposed architecture should be easy to modify and expand, adding or modifying input channels and changing the multimodal fusion strategies. In this paper, we introduce our general approach and provide a case study with two modalities (gesture and speech).

We present a robotic control system endowed with attentional mechanisms suitable for balancing the trade off between safe human-robot interaction and effective task execution. These mechanisms allow the robot to increase or decrease the degree of attention toward relevant activities modulating the frequency of the monitoring rate and the speed associated to the robot movements. In this framework, we consider pick-and-place and give-and-receive attentional behaviors.

In this paper, we propose a human-robot interaction system that exploits emotion and attention to regulate and adapt the robotic interactive behavior. In particular, we will focus on the relation between arousal, predictability, and attentional allocation considering as a case study a robotic manipulator interacting with a human operator. We rely on a frequency based model of attention allocation and a 4-dimensional model of emotion. The experiment reported in this paper explores the effectiveness of an attentional regulation mechanisms modulated by arousal and predictability values extracted from the human voice. The collected results show that the attentional modulation, mediated by basic emotional speech features, provides a natural and computationally light regulation mechanism for coordinating the robotic behaviors.

Timeline-based planning techniques have demonstrated wide application possibilities in heterogeneous real world domains. For a wider diffusion of this technology, a more thorough investigation of the connections with formal methods is needed. This pa

Timeline-based planners have been shown quite successful in addressing real world problems. Nevertheless they are considered as a niche technology in AI P&S research as an application synthesis with such techniques is still considered a sort of “black art”. Authors are currently developing a knowledge engineering tool around a timeline-based problem solving environment; in this framework we aim at integrating verification and validation methods. This work presents a verification process suitable for a timeline-based planner. It shows how a problem of flexible temporal plan verification can be cast as model-checking on timed game automata. Additionally it provides formal properties and checks the effectiveness of the proposed approach with a detailed experimental analysis.

In this work we describe a model-based approach to the executive control of a rescue rover. We show how this control architecture naturally supports human-robot interaction in the diverse activities needed in rescue and search. We illustrate the approach by considering human-robot interaction in the domain of the RoboCup rescue competition. We discuss the implementation and tests done both during RoboCup contests and in the laboratory, to show performances according to different working modalities such as fully operated, supervised, fully autonomous.

This paper reports on the design and implementation of a real-time executive for a mobile rover that uses a model-based, declarative approach. The control system is based on the intelligent distributed execution architecture (IDEA), an approach to planning and execution that provides a unified representational and computational framework for an autonomous agent. The basic hypothesis of IDEA is that a large control system can be structured as a collection of interacting agents, each with the same fundamental structure. We show that planning and real-time response are compatible if the executive minimizes the size of the planning problem. We detail the implementation of this approach on an 'exploration rover (Gromit, an RWI ATRV Junior at NASA Ames) presenting different IDEA controllers of the same domain and comparing them with more classical approaches. We demonstrate that the approach is scalable to complex coordination of functional modules needed for autonomous navigation and exploration.