Despite the spread of technologies in the physical world and the normalization of virtual experiences, non-verbal communication with radically non-anthropomorphic avatars remains an underexplored frontier. We present an interaction system in which two participants must learn to communicate with each other non-verbally through a digital filter that morphs their appearance. In a collaborative escape room, the Visitor must teach a non-anthropomorphic physical robot to play, while the Controller, in a different location, embodies the robot with an altered perception of the environment and the Visitor’s companion in VR. This study addresses the design of the activity, the robot, and the virtual environment, with a focus on how the Visitor’s morphology is translated in VR. Results show that participants were able to develop emergent and effective communication strategies, with the Controller naturally embodying its avatar’s narrative, making this system a promising testbed for future research on human-technology interaction, entertainment, and embodiment.

We are investigating interaction with entities showing features different from humans', to understand how they can be embodied as avatars and perceived as living, social beings. To push this investigation to its limit, we have designed as an avatar an interactive space (the Room), that challenges both the anthropomorphic structure, and most of the social interaction mechanisms we are used to. We introduce a first framework for the Room design, addressing challenges related to its body, perception, and interaction process. We present a pilot implementation of some of the aspects of the framework as an interactive installation, namely a real-time, two-player, VR experience, featuring the Room avatar, with a focus on haptic feedback as the main means of perception for the subject embodying the Room. By radically challenging anthropomorphism, we seek to investigate the most basic aspects of embodiment and social cognition.

MushroomRL is an open-source Python library developed to simplify the process of implementing and running Reinforcement Learning (RL) experiments. Compared to other available libraries, MushroomRL has been created with the purpose of providing a comprehensive and flexible framework to minimize the effort in implementing and testing novel RL methodologies. The architecture of MushroomRL is built in such a way that every component of a typical RL experiment is already provided, and most of the time users can only focus on the implementation of their own algorithms. MushroomRL is accompanied by a benchmarking suite collecting experimental results of state-of-the-art deep RL algorithms, and allowing to benchmark new ones. The result is a library from which RL researchers can significantly benefit in the critical phase of the empirical analysis of their works. MushroomRL stable code, tutorials, and documentation can be found at https://github.com/MushroomRL/mushroom-rl.

In the field of reproductive technology, the biochemical composition of female gametes has been successfully investigated with the use of vibrational spectroscopy. Currently, in assistive reproductive technology (ART), there are no shared criteria for the choice of oocyte, and automatic classification methods for the best quality oocytes have not yet been applied. In this paper, considering the lack of criteria in Assisted Reproductive Technology (ART), we use Machine Learning (ML) techniques to predict oocyte quality for a successful pregnancy. To improve the chances of successful implantation and minimize any complications during the pregnancy, Fourier transform infrared microspectroscopy (FTIRM) analysis has been applied on granulosa cells (GCs) collected along with the oocytes during oocyte aspiration, as it is routinely done in ART, and specific spectral biomarkers were selected by multivariate statistical analysis. A proprietary biological reference dataset (BRD) was successfully collected to predict the best oocyte for a successful pregnancy. Personal health information are stored, maintained and backed up using a cloud computing service. Using a user-friendly interface, the user will evaluate whether or not the selected oocyte will have a positive result. This interface includes a dashboard for retrospective analysis, reporting, real-time processing, and statistical analysis. The experimental results are promising and confirm the efficiency of the method in terms of classification metrics: precision, recall, and F1-score (F1) measures.

A key issue for a device involved in a competitive game is to be able to match the ability of the opponent, thus making the game interesting. When the game is played by a human player against a machine, this should be able to adapt online to the opponent's ability, which may change during the game. When the playing machine is a robot, and the game involves physical activity, adaptation should consider the player's ability, but also other aspects directly related to the physical nature of the game. Adaptation could be obtained either intrinsically, by designing the interaction, or explicitly, by modeling aspects of the opponent that can enable to estimate performance, behaviors, and strategies. We present in this paper different approaches to design the two types of adaptation, together with two competitive, physically interactive robogames exploiting them.

There is an increasing interest in Reinforcement Learning to solve new and more challenging problems, as those emerging in robotics and unmanned autonomous vehicles. To face these complex systems, a hierarchical and multi-scale representation is crucial. This has brought the interest on Hierarchical Deep Reinforcement learning systems. Despite their successful application, Deep Reinforcement Learning systems suffer from a variety of drawbacks: they are data hungry, they lack of interpretability, and it is difficult to derive theoretical properties about their behavior. Classical Hierarchical Reinforcement Learning approaches, while not suffering from these drawbacks, are often suited for finite actions, and finite states, only. Furthermore, in most of the works, there is no systematic way to represent domain knowledge, which is often only embedded in the reward function. We present a novel Hierarchical Reinforcement Learning framework based on the hierarchical design approach typical of control theory. We developed our framework extending the block diagram representation of control systems to fit the needs of a Hierarchical Reinforcement Learning scenario, thus giving the possibility to integrate domain knowledge in an effective hierarchical architecture.

Social robots are now being considered to be a part of the therapy of children with autism. During the interactions, some aggressive behaviors could lead to harmful scenarios. The ability of a social robot to detect such behaviors and react to intervene or to notify the therapist would improve the outcomes of therapy and prevent any potential harm toward another person or to the robot. In this study, we investigate the feasibility of an artificial neural network in classifying 6 interaction behaviors between a child and a small robotic toy. The behaviors were: hit, shake, throw, pickup, drop, and no interaction or idle. Due to the ease of acquiring data from adult participants, a model was developed based on adults' data and was evaluated with children's data. The developed model was able to achieve promising results based on the accuracy (i.e. 80%), classification report (i.e. overall F1-score=80%), and confusion matrix. The findings highlight the possibility of characterizing children's negative interactions with robotic toys to improve safety.

In the Multiagent Connected Path Planning problem (MCPP), a team of agents moving in a graph-represented environment must plan a set of start-goal joint paths which ensures global connectivity at each time step, under some communication model. The decision version of this problem asking for the existence of a plan that can be executed in at most a given number of steps is claimed to be NP-complete in the literature. The NP membership proof, however, is not detailed. In this paper, we show that, in fact, even deciding whether a feasible plan exists is a PSPACE-complete problem. Furthermore, we present three algorithms adopting different search paradigms, and we empirically show that they may efficiently obtain a feasible plan, if any exists, in different settings.

We propose a quantitative human player model for Physically Interactive RoboGames that can account for the combination of the player activity (physical effort) and interaction level. The model is based on activity recognition and a description of the player interaction (proximity and body contraction index) with the robot co-player. Our approach has been tested on a dataset collected from a real, physical robot game, where activity patterns extracted by a custom 3-axis accelerometer sensor module and by the Microsoft Kinect sensor are used. The proposed model design aims at inspiring approaches that can consider the activity of a human player in lively games against robots and foster the design of robotic adaptive behavior capable of supporting her/his engagement in such type of games.

Our research explores social robots as learning tools for persons with Neurodevelopmental Disorder (NDD). The paper reports an empirical study that investigates engagement as a prerequisite for any learning process of NDD subjects. The study involved 5 persons in this target group and three robots (two research products developed at our lab, and a commercial one), which were used in sequence during individual therapeutic sessions at a care center. The results enable us to compare the engagement effects of different social robots and improves our understanding of the behavior of persons with NDD during robotic experiences.

Emotions are considered by many researchers as beneficial elements in social robotics, since they can enrich human-robot interaction. Although there have been works that have studied emotion expression in robots, mechanisms to express emotion are usually highly integrated with the rest of the system. This limits the possibility to use these approaches in other applications. This paper presents a system that has been initially created to facilitate the study of emotion projection, but it has been designed to enable its adaptation in other fields. The emotional enrichment system has been envisioned to be used with any action decision system. A description of the system components and their characteristics are provided. The system has been adapted to two different platforms with different degrees of freedom: Keepon and Triskarino.

We propose a new emotional, huggable, mobile, and configurable robot (Teo), which can address some of the still open therapeutic needs in the treatment of Developmental Disability (DD). Teo has been designed in partnership with a team of DD specialists, and it is meant to be used as an efficient and easy-to-use tool for caregivers. Teo is integrated with virtual worlds shown on large displays or projections and with external motion sensing devices to support various forms of full-body interaction and to engage DD persons in a variety of play activities that blend the digital and physical world and can be fully customized by therapists to meet the requirements of each single subject. Exploratory studies have been performed at two rehabilitation centres to investigate the potential of our approach. The positive results of these studies pinpoint that our system endeavors promising opportunities to offer new forms of interventions for DD people.

The core contribution of this paper lies in exploring new spaces of interaction for children with Intellectual Developmental Disorder (IDD). In the KROG (Kinect-RObot for Gaming) Project, we blend full-body interaction, virtual worlds on large screens, motion sensing technology, and mobile robots to support game-based interventions. This paper highlights the design challenges induced by this mix of technologies and interaction paradigms, presents the prototypes that have been iteratively designed and tested with 22 specialists, and discusses the lessons learned from our project.

Our research explores the integration of motion-based touchless interaction with human-robots interaction to support game-based learning for children with intellectual disability. The paper discusses the design challenges of this novel approach and presents the design concepts of our initial prototypes.

Studies about human interaction have shown that subtle changes in movement performance and body posture may improve people's acceptance in social groups. The same applies also to robots. However, most of the work has been done on faces and bio-inspired or humanoid robots, while still few works have focused on generic robot body movement to produce interesting interaction settings, which include emotion projection. These studies have to take into account the restrictions and peculiarities of robots, like degrees of freedom, or a body shape designed to achieve a specific functionality - such as, for instance, vacuum cleaning - and not necessarily bio-inspired. This work reports the results obtained from two experiments performed to study whether features different from face and bio-inspired bodies could convey emotions. The study has been done with a non-bio-inspired robot base, having an intentionally unusual (for a robot) shape. The results show that it is possible to convey emotions using features that can be implemented also in a non-bio-inspired embodiment.

Theatre, movies, and TV series captivate people because they show interesting stories. The success of these stories does not depend exclusively on their script, but also on the realism that actors incorporate in their performance. This realism means that actors must reflect everyday human-human interactions. However, theatre demands something that movies and TV series do not, that is the live production of believable human-human interaction. Theatre actors do not have second chances to recover problems in their performance with the same audience. They have to project emotions to the whole audience to make them believe in the played character and to engage them in the play: this same principle is used in effective social relations played in the real world. The aim of the project introduced in this paper is to implement a theatrical robot actor that can perform on the stage with human actors, generating the appropriate emotional expressions and social behavior, and solving autonomously eventual problems that could rise in the representation. Moreover, it should have a simple interface to enable untrained people, such as a director, to give it basic instructions that it can interpret to effectively play its role in the piece. Theatrical robots developed so far could mainly be used as props in theatre, and do not exploit theatre's constrains, nor could express emotions automatically generated from text and directions. The development of a theatrical actor is a first step towards the implementation of effective autonomous robots able to socially interact with people, the system and platform could be extended to other areas where showing emotions is important, as in robot games and assistive robots.

In this paper we address the combination of batch reinforcement-learning (BRL) techniques with direct policy search (DPS) algorithms in the context of robot learning. Batch value-based algorithms (such as fitted Q-iteration) have been proved to outperform online ones in many complex applications, but they share the same difficulties in solving problems with continuous action spaces, such as robotic ones. In these cases, actor-critic and DPS methods are preferable, since the optimization process is limited to a family of parameterized (usually smooth) policies. On the other hand, these methods (e.g., policy gradient and evolutionary methods) are generally very expensive, since finding the optimal parameterization may require to evaluate the performance of several policies, which in many real robotic applications is unfeasible or even dangerous. To overcome such problems, we exploit the fitted policy search (FPS) approach, in which the expected return of any policy considered during the optimization process is evaluated offline (without resorting to the robot) by reusing the data collected in the initial exploration phase. In this way, it is possible to take the advantages of both BRL and DPS algorithms, thus achieving an effective learning approach to solve robotic problems. A balancing task on a real two-wheeled robotic pendulum is used to analyze the properties and evaluate the effectiveness of the FPS approach.

DataMining is most commonly used in attempts to induce association rules from transaction data which can help decision-makers easily analyze the data and make good decisions regarding the domains concerned. Most conventional studies are focused on binary or discrete-valued transaction data, however the data in real-world applications usually consists of quantitative values. In the last years, many researches have proposed Genetic Algorithms for mining interesting association rules from quantitative data. In this paper, we present a study of three genetic association rules extraction methods to show their effectiveness for mining quantitative association rules. Experimental results over two real-world databases are showed.

We present a method to diagnose lung cancer by the analysis of breath using an electronic nose. This device can react to a gas substance by providing signals that can be analyzed to classify the input. It is composed of a sensor array (6 MOS sensors, in our case) and a pattern classification process based on machine learning techniques. During the first phase of our research, we have evaluated the possibility and accuracy of lung cancer diagnosis by classifying the olfactory signal associated to exhalations of subjects. The second part of the research, still in progress, is aimed at assessing the possibility of discriminating also the different types and stages of the disease. At the end of the first phase, results have been very satisfactory and promising: we achieved an average accuracy of 92.6%, sensitivity of 95.3% and specificity of 90.5%. In particular we analyzed the breath of 101 individuals, of which 58 control subjects, and 43 suffer from different types of lung cancer (primary and not) at different stages. In order to find the components able to discriminate between the two classes ‘healthy’ and ‘sick’ at best, and to reduce the dimensionality of the problem, we have extracted the most significant features and projected them into a lower dimensional space using Non Parametric Linear Discriminant Analysis. Finally, we have used these features as input to several supervised pattern classification algorithms, based on different k-nearest neighbors (k-NN) approaches (classic, modified and Fuzzy k-NN), linear and quadratic discriminant classifiers and on a feed-forward artificial neural network (ANN). The observed results have all been validated using cross-validation. These results pushed us to begin the second phase of the project to investigate the possibility of early lung cancer diagnosis: we are involving a larger number of subjects, partioned in different classes according to the type and stage of the disease. The research demonstrates that the electronic nose is a promising alternative to current lung cancer diagnostic techniques: the obtained predictive errors are lower than those achieved by present diagnostic methods, and the cost of the analysis, both in money, time and resources, is lower. The introduction of this technology will lead to very important social and business effects: its low price and small dimensions allow a large scale distribution, giving the opportunity to perform non invasive, cheap, quick, and massive early diagnosis and screening.

The main objective of transfer in reinforcement learning is to reduce the complexity of learning the solution of a target task by effectively reusing the knowledge retained from solving a set of source tasks. In this paper, we introduce a novel algorithm that transfers samples (i.e., tuples 〈s, a, s', r〉) from source to target tasks. Under the assumption that tasks have similar transition models and reward functions, we propose a method to select samples from the source tasks that are mostly similar to the target task, and, then, to use them as input for batch reinforcement-learning algorithms. As a result, the number of samples an agent needs to collect from the target task to learn its solution is reduced. We empirically show that, following the proposed approach, the transfer of samples is effective in reducing the learning complexity, even when some source tasks are significantly different from the target task.

We present FIXCS (fuzzy implementation of XCS), a learning classifier system that extend the accuracy-based extended classifier system (XCS) by allowing to match real-valued input by fuzzy sets, and to produce a fuzzy output, then translated into real values. This work gives XCS the ability to face real-valued problems with a fuzzy model that approximates a real valued function better than the original, interval-based model. First results show that, as expected, the learning time is longer, but the obtained fuzzy system is more robust than the interval-based one.

We present a method to recognize the presence of lung cancer in individuals by classifying the olfactory signal acquired through an electronic nose based on an array of MOS sensors. We analyzed the breath of 101 persons, of which 58 as control and 43 suffering from different types of lung cancer (primary and not) at different stages. In order to find the components able to discriminate between the two classes 'healthy' and 'sick' as best as possible and to reduce the dimensionality of the problem, we extracted the most significative features and projected them into a lower dimensional space, using Non Parametric Linear Discriminant Analysis. Finally, we used these features as input to several supervised pattern classification techniques, based on different k-nearest neighbors (k-NN) approaches (classic, modified and Fuzzy k-NN), linear and quadratic discriminant classifiers and on a feedforward artificial neural network (ANN). The observed results, all validated using cross-validation, have been satisfactory, achieving an accuracy of 92.6%, a sensitivity of 95.3% and a specificity of 90.5%. These results put the electronic nose as a valid implementation of lung cancer diagnostic technique, being able to obtain excellent results with a non invasive, small, low cost and very fast instrument.

Learning in real-world domains often requires to deal with continuous state and action spaces. Although many solutions have been proposed to apply Reinforce- ment Learning algorithms to continuous state problems, the same techniques can be hardly extended to continuous action spaces, where, besides the computation of a good approximation of the value function, a fast method for the identification of the highest-valued action is needed. In this paper, we propose a novel actor-critic approach in which the policy of the actor is estimated through sequential Monte Carlo methods. The importance sampling step is performed on the basis of the values learned by the critic, while the resampling step modifies the actor’s policy. The proposed approach has been empirically compared to other learning algo- rithms into several domains; in this paper, we report results obtained in a control problem consisting of steering a boat across a river.

In this paper, we present a dead reckoning sensor to support reliable odometry on mobile robots. This sensor is based on a pair of optical mice rigidly connected to the robot body and its main advantages are 1) this localization system is independent from the kinematics of the robot, 2) the measurement given by the mice is not subject to slipping, since they are independent from the traction wheels, nor to crawling, since they measure displacements in any direction 3) it is a low-cost solution with a precision comparable to classical shaft encoders. Since we have redundant measures it is possible to detect non-systematic errors; in this paper, an automatic procedure to reduce non-systematic errors of the sensor is presented and validated with experimental results on a real mobile robot.

In this paper, we present a dead reckoning sensor to support reliable odometry on mobile robots. This sensor is based on a pair of optical mice rigidly connected to the robot body and its main advantages are 1) this localization system is independent from the kinematics of the robot, 2) the measurement given by the mice is not subject to slipping, since they are independent from the traction wheels, nor to crawling, since they measure displacements in any direction 3) it is a low-cost solution with a precision comparable to classical shaft encoders. We present the mathematical model of the sensor, its implementation, and some experimental evaluations using the standard UMBmark benchmark for odometry.

The behavior of agents in complex and dynamic environments cannot be programmed a priori, but needs to self-adapt to the specific situations. We present some approaches based on evolutionary reinforcement learning algorithms, which are able to evolve in real-time fuzzy models that control behaviors. We discuss an application where an agent learns how to adapt its behavior to the different behaviors of the other agents it is interacting with, and another application where a group of agents co-evolve cooperative behaviors by using explicit communication to propose the cooperation and to distribute reinforcement to the others.

The success of any reinforcement learning (RL) application is in large part due to the design of an appropriate reinforcement function. A methodological framework to support the design of reinforcement functions has not been defined yet, and this critical and often underestimated activity is left to the ability of the RL application designer. We propose an approach to support reinforcement function design in RL applications concerning learning behaviors for autonomous agents. We define some dimensions along which we can describe reinforcement functions; we consider the distribution of reinforcement values, their coherence and their matching with the designer's perspective. We give hints to define measures that objectively describe the reinforcement function; we discuss the trade-offs that should be considered to improve learning and we introduce the dimensions along which this improvement can be expected. The approach we are presenting is general enough to be adopted in a large number of RL projects. We show how to apply it in the design of learning classifier systems (LCS) applications. We consider a simple, but quite complete case study in evolutionary robotics, and we discuss reinforcement function design issues in this sample context.

Models that are constructed within the bounds of a single paradigm are not sufficient for modeling all aspects of complex systems. Therefore, even though reasoning and simulation systems that utilize a single modeling paradigm are the current norm, we explore a multimodel approach in this paper. A multimodel approach is defined as one in which more than one model-each derived from a different perspective, and utilizing correspondingly distinct reasoning and simulation strategies-are employed. By describing four models which illustrate the use of different modeling techniques, we show how a multimodel approach can enrich the modeling environment and make it correspond better with real world information. Our models come from many sources-Systems and Simulation literature for the modeling of natural phenomena and artificial devices, and Artificial Intelligence and Cognitive Science for the modeling of human intuition and expertise in reasoning. Generalizing from these four models, we suggest that modeling complex systems may best be approached from an integrated architectural viewpoint which combines multiple modeling paradigms.>

The architecture of an uncertainty manager (UM) based on belief revision concepts is proposed. It automatically maintains the consistency of related propositions according to a given theory of uncertainty. Facts are represented as nodes of a network, whose links stand for inferential dependencies among the connected nodes. Each node is associated to a measure of the confidence the UM has in it. The effect of any change in the network is propagated throughout the whole network to preserve consistency. The motivations to accept cyclic dependencies among propositions in such a network are discussed, and the solutions to the problems arising are presented. A theory of uncertainty suitable for representing subjective linguistic estimates is proposed as a solution to the representation of uncertainty coming from subjective judgements. The UM can work with any uncertainty theory whose definition satisfies the general framework presented.>