Two of the main challenges in multi-label classification are the need to collect labeled data, which can be costly or impractical, and the need to satisfy hard logical constraints between labels, which is often computationally expensive.In some applications, complementary labelsthat is, labels specifying a class to which a sample does not belong -are available and much less costly to obtain.Researchers have therefore developed methods to learn from such labels efficiently and effectively.Similar efforts have been made to address the problem of learning with hard logical constraints.Nevertheless, to the best of our knowledge, no prior work has investigated the problem of learning from complementary labels with hard logical constraints.In this work, we propose and compare methods to address this problem, showing that hard logical constraints, besides representing restrictions to be satisfied, can also serve as an additional source of weak supervision.The relationships between labels can help bridge the information gap between relevant and complementary labels.Experimental results on different datasets and scenarios support our claims.

As the primary messenger RNA (mRNA) delivery vehicles, ionizable lipid nanoparticles (LNPs) exhibit excellent safety, high transfection efficiency, and strong immune response induction. However, the screening process for LNPs is time-consuming and costly. To expedite the identification of high-transfection-efficiency mRNA drug delivery systems, we propose an explainable LNPs transfection efficiency prediction model, called TransMA. TransMA employs a multimodal molecular structure fusion architecture, wherein the fine-grained atomic spatial relationship extractor named molecule 3D Transformer captures three-dimensional spatial features of the molecule, and the coarse-grained atomic sequence extractor named molecule Mamba captures one-dimensional molecular features. We design the mol-attention mechanism block, enabling it to align coarse and fine-grained atomic features and capture relationships between atomic spatial and sequential structures. TransMA achieves state-of-the-art performance in predicting transfection efficiency using the scaffold and cliff data splitting methods on the current largest LNPs dataset, including Hela and RAW cell lines. Moreover, we find that TransMA captures the relationship between subtle structural changes and significant transfection efficiency variations, providing valuable insights for LNPs design. Additionally, TransMA's predictions on external transfection efficiency data maintain a consistent order with actual transfection efficiencies, demonstrating its robust generalization capability. We hope that high-accuracy transfection prediction models in the future can aid in LNPs design and initial screening, thereby assisting in accelerating the mRNA design process.

This work aims to tackle the intent recognition problem in Human-Robot Collaborative assembly scenarios. Precisely, we consider an interactive assembly of a wooden stool where the robot fetches the pieces in the correct order and the human builds the parts following the instruction manual. The intent recognition is limited to the idle state estimation and it is needed to ensure a better synchronization between the two agents. We carried out a comparison between two distinct solutions involving wearable sensors and eye tracking integrated into the perception pipeline of a flexible planning architecture based on Hierarchical Task Networks. At runtime, the wearable sensing module exploits the raw measurements from four 9-axis Inertial Measurement Units positioned on the wrists and hands of the user as an input for a Long Short-Term Memory Network. On the other hand, the eye tracking relies on a Head Mounted Display and Unreal Engine.

This paper presents a comprehensive framework to enhance Human-Robot Collaboration (HRC) in real-world scenarios. It introduces a formalism to model articulated tasks, requiring cooperation between two agents, through a smaller set of primitives. Our implementation leverages Hierarchical Task Networks (HTN) planning and a modular multisensory perception pipeline, which includes vision, human activity recognition, and tactile sensing. To showcase the system’s scalability, we present an experimental scenario where two humans alternate in collaborating with a Baxter robot to assemble four pieces of furniture with variable components. This integration highlights promising advancements in HRC, suggesting a scalable approach for complex, cooperative tasks across diverse applications.

Dexterous in-hand manipulation is a unique and valuable human skill requiring sophisticated sensorimotor interaction with the environment while respecting stability constraints. Satisfying these constraints with generated motions is essential for a robotic platform to achieve reliable in-hand manipulation skills. Explicitly modelling these constraints can be challenging, but they can be implicitly modelled and learned through experience or human demonstrations. We propose a learning and control approach based on dictionaries of motion primitives generated from human demonstrations. To achieve this, we defined an optimization process that combines motion primitives to generate robot fingertip trajectories for moving an object from an initial to a desired final pose. Based on our experiments, our approach allows a robotic hand to handle objects like humans, adhering to stability constraints without requiring explicit formalization. In other words, the proposed motion primitive dictionaries learn and implicitly embed the constraints crucial to the in-hand manipulation task.

As collaborative robots become more common in manufacturing scenarios and adopted in hybrid human-robot teams, we should develop new interaction and communication strategies to ensure smooth collaboration between agents. In this paper, we propose a novel communicative interface that uses Mixed Reality as a medium to perform Kinesthetic Teaching (KT) on any robotic platform. We evaluate our proposed approach in a user study involving multiple subjects and two different robots, comparing traditional physical KT with holographic-based KT through user experience questionnaires and task-related metrics.

This article investigates mixed reality (MR) to enhance human-robot collaboration (HRC). The proposed solution adopts MR as a communication layer to convey a mobile manipulator’s intentions and upcoming actions to the humans with whom it interacts, thus improving their collaboration. A user study involving 20 participants demonstrated the effectiveness of this MR-focused approach in facilitating collaborative tasks, with a positive effect on overall collaboration performances and human satisfaction.

Nowadays, the most adopted technique to address infertility problems is in vitro fertilisation (IVF). However, its success rate is limited, and the associated procedures, known as assisted reproduction technology (ART), suffer from a lack of objectivity at the laboratory level and in clinical practice. This paper deals with applications of Artificial Intelligence (AI) techniques to IVF procedures. Artificial intelligence is considered a promising tool for ascertaining the quality of embryos, a critical step in IVF. Since the oocyte quality influences the final embryo quality, we present a systematic review of the literature on AI-based techniques used to assess oocyte quality; we analyse its results and discuss several promising research directions. In particular, we highlight how AI-based techniques can support the IVF process and examine their current applications as presented in the literature. Then, we discuss the challenges research must face in fully deploying AI-based solutions in current medical practice. Among them, the availability of high-quality data sets as well as standardised imaging protocols and data formats, the use of physics-informed simulation and machine learning techniques, the study of informative, descriptive yet observable features, and, above all, studies of the quality of oocytes and embryos, specifically about their live birth potential. An improved understanding of determinants for oocyte quality can improve success rates while reducing costs, risks for long-term embryo cultures, and bioethical concerns.

Hands are a fundamental tool humans use to interact with the environment and objects. Through hand motions, we can obtain information about the shape and materials of the surfaces we touch, modify our surroundings by interacting with objects, manipulate objects and tools, or communicate with other people by leveraging the power of gestures. For these reasons, sensorized gloves, which can collect information about hand motions and interactions, have been of interest since the 1980s in various fields, such as human–machine interaction and the analysis and control of human motions. Over the last 40 years, research in this field explored different technological approaches and contributed to the popularity of wearable custom and commercial products targeting hand sensorization. Despite a positive research trend, these instruments are not widespread yet outside research environments and devices aimed at research are often ad hoc solutions with a low chance of being reused. This article aims to provide a systematic literature review for custom gloves to analyze their main characteristics and critical issues, from the type and number of sensors to the limitations due to device encumbrance. The collection of this information lays the foundation for a standardization process necessary for future breakthroughs in this research field.

In this paper, we look into the minimum obstacle displacement (MOD) planning problem from a mobile robot motion planning perspective. This problem finds an optimal path to goal by displacing movable obstacles when no path exists due to collision with obstacles. However this problem is computationally expensive and grows exponentially in the size of number of movable obstacles. This work looks into approximate solutions that are computationally less intensive and differ from the optimal solution by a factor of the optimal cost.

In this paper, we propose a new vision system for marine rescue drones called Falcon. Our system is designed to track and monitor individuals in need of rescue using a wide-angle field of view and a gimbal mechanism to control the camera direction. Additionally, our system utilizes a high-resolution central field of view to assess the priority level of the person in need of rescue. We conducted two experiments to test our system's performance. The first experiment focused on the correlation between the data compression rate of the system and the precision and recall scores of human detection using YOLO in three different flight altitudes. The second experiment examined the correlation between the distance from the center of the images captured by the system and the confidence score of the detected persons, considering the resolution difference from the central region to the periphery.

Implicit communication plays such a crucial role during social exchanges that it must be considered for a good experience in human-robot interaction. This work addresses implicit communication associated with the detection of physical properties, transport, and manipulation of objects. We propose an ecological approach to infer object characteristics from subtle modulations of the natural kinematics occurring during human object manipulation. Similarly, we take inspiration from human strategies to shape robot movements to be communica-tive of the object properties while pursuing the action goals. In a realistic HRI scenario, participants handed over cups - filled with water or empty - to a robotic manipulator that sorted them. We implemented an online classifier to differentiate careful/not careful human movements, associated with the cups' content. We compared our proposed “expressive” controller, which modulates the movements according to the cup filling, against a neutral motion controller. Results show that human kinematics is adjusted during the task, as a function of the cup content, even in reach-to-grasp motion. Moreover, the carefulness during the handover of full cups can be reliably inferred online, well before action completion. Finally, although questionnaires did not reveal explicit preferences from partici-pants, the expressive robot condition improved task efficiency.

This article presents an open-source architecture for conveying robots’ intentions to human teammates using Mixed Reality and Head-Mounted Displays. The architecture has been developed focusing on its modularity and re-usability aspects. Both binaries and source code are available, enabling researchers and companies to adopt the proposed architecture as a standalone solution or to integrate it in more comprehensive implementations. Due to its scalability, the proposed architecture can be easily employed to develop shared Mixed Reality experiences involving multiple robots and human teammates in complex collaborative scenarios.

The possibility for humans to interact with physical or virtual systems using gestures has been vastly explored by researchers and designers in the last 20 years to provide new and intuitive interaction modalities. Unfortunately, the literature about gestural interaction is not homogeneous, and it is characterized by a lack of shared terminology. This leads to fragmented results and makes it difficult for research activities to build on top of state-of-the-art results and approaches. The analysis in this article aims at creating a common conceptual design framework to enforce development efforts in gesture-based human-machine interaction (HMI). The main contributions of this article can be summarized as follows: 1) we provide a broad definition for the notion of functional gesture in HMI; 2) we design a flexible and expandable gesture taxonomy; and 3) we put forward a detailed problem statement for gesture-based HMI. Finally, to support our main contribution, this article presents and analyzes 83 most pertinent articles classified on the basis of our taxonomy and problem statement.

Humans engaged in collaborative activities are naturally able to convey their intentions to teammates through multi-modal communication, which is made up of explicit and implicit cues. Similarly, a more natural form of human-robot collaboration may be achieved by enabling robots to convey their intentions to human teammates via multiple communication channels. In this paper, we postulate that a better communication may take place should collaborative robots be able to anticipate their movements to human teammates in an intuitive way. In order to support such a claim, we propose a robot system's architecture through which robots can communicate planned motions to human teammates leveraging a Mixed Reality interface powered by modern head-mounted displays. Specifically, the robot's hologram, which is superimposed to the real robot in the human teammate's point of view, shows the robot's future movements, allowing the human to understand them in advance, and possibly react to them in an appropriate way. We conduct a preliminary user study to evaluate the effectiveness of the proposed anticipatory visualization during a complex collaborative task. The experimental results suggest that an improved and more natural collaboration can be achieved by employing this anticipatory communication mode.

Even though cobots have high potential in bringing several benefits in the manufacturing and logistic processes, their rapid (re-)deployment in changing environments is still limited. To enable fast adaptation to new product demands and to boost the fitness of the human workers to the allocated tasks, we propose a novel method that optimizes assembly strategies and distributes the effort among the workers in human-robot cooperative tasks. The cooperation model exploits AND/OR Graphs that we adapted to solve also the role allocation problem. The allocation algorithm considers quantitative measurements that are computed online to describe human operators' ergonomic status and task properties. We conducted preliminary experiments to demonstrate that the proposed approach succeeds in controlling the task allocation process to ensure safe and ergonomic conditions for the human worker.

In this paper, the authors present Sidewinder, a unique stereo vision system for snake-shaped robots operating in search and rescue scenarios. The robot should navigate the environment by finding and passing through narrow spaces (i.e., forward vision task) and search for persons in need of rescue (i.e., panoramic vision task). Therefore, we propose two types of image mapping methods to generate respectively input images for stereo visual SLAM and person detection. This work presents preliminary results using Sidewinder's images as input for the ORS-SLAM2 for localization and mapping, and YOLO-v3, for person detection.

Dexterous in-hand manipulation is a peculiar and useful human skill. This ability requires the coordination of many senses and hand motion to adhere to many constraints. These constraints vary and can be influenced by the object characteristics or the specific application. One of the key elements for a robotic platform to implement reliable in-hand manipulation skills is to be able to integrate those constraints in their motion generations. These constraints can be implicitly modelled, learned through experience or human demonstrations. We propose a method based on motion primitives dictionaries to learn and reproduce in-hand manipulation skills. In particular, we focused on fingertip motions during the manipulation, and we defined an optimization process to combine motion primitives to reach specific fingertip configurations. The results of this work show that the proposed approach can generate manipulation motion coherent with the human one and that manipulation constraints are inherited even without an explicit formalization.

We present Arianna+, a framework to design networks of ontologies for representing knowledge enabling smart homes to perform human activity recognition online. In the network, nodes are ontologies allowing for various data contextualisation, while edges are general-purpose computational procedures elaborating data. Arianna+provides a flexible interface between the inputs and outputs of procedures andstatements, which are atomic representations of ontological knowledge. Arianna+schedules procedures on the basis ofeventsby employing logic-based reasoning, that is, by checking the classification of certain statements in the ontologies. Each procedure involves input and output statements that are differently contextualized in the ontologies based on specific prior knowledge. Arianna+allows to design networks that encode data within multiple contexts and, as a reference scenario, we present a modular network based on a spatial context shared among all activities and a temporal context specialized for each activity to be recognized. In the article, we argue that a network of small ontologies is more intelligible and has a reduced computational load than a single ontology encoding the same knowledge. Arianna+integrates in the same architecture heterogeneous data processing techniques, which may be better suited to different contexts. Thus, we do not propose a new algorithmic approach to activity recognition, instead, we focus on the architectural aspects for accommodating logic-based and data-driven activity models in a context-oriented way. Also, we discuss how to leverage data contextualization and reasoning for activity recognition, and to support an iterative development process driven by domain experts.

This paper presents a method of wide angle fovea visual odometry (WAF-VO) for Wide Angle Fovea Sensor SLAM (WAF-SLAM), by which a unique locally-high accurate and wide-angle map is generated in addition to camera motion estimation. The WAF sensor is a special-made wide-angle sensor that is inspired from human visual function, i.e., the spatial resolution of the image is not uniform throughout the entire field of view (FOV); it is much higher in the central FOV and decreases rapidly towards the periphery. Our visual odometry method is strongly characterized by a wide-angle FOV and space-variant resolution of the input image from the WAF sensor. A locally-high accurate and wide-angle mapping method is proposed as a major part for WAF-SLAM together with the camera motion estimation. Our proposed method estimates camera motions more stably using very low-spatial-resolution wide-angle images remapped from the input image of the WAF sensor. Using the estimated camera motions, narrow-angle high accurate maps are generated from corresponding feature points in high-spatial resolution central regions of the input image. Wide-angle maps are generated from ones in middle-spatial-resolution wide-angle images remapped from the input image apart from the above very low-spatial-resolution images. When the wide-angle maps are generated, the number of extracted feature points is increased by adjusting contrast threshold values of SIFT feature according to regions of the FOV. A KNN matching method improved using epipolar constraint is proposed and employed for avoidance of mismatching the increased feature points. Thus, the wide-angle maps are generated from more correct corresponding feature points. Finally, the above two types of maps are combined into the unique locally-high accurate and wide-angle map, i.e., a WAF map. Using our proposed method, the WAF map was generated by verification experiments. Furthermore, the paper presents an evaluation of the accuracy and precision of the generated map.

In this work, we present a framework for human-robot collaboration allowing the human operator to alter the robot plan execution online. To achieve this goal, we introduce Branched AND/OR graphs, an extension to AND/OR graphs, to manage flexible and adaptable human-robot collaboration. In our study, the operator can alter the plan execution using two implementations of Branched AND/OR graphs for learning by demonstration, using kinesthetic teaching, and task repetition. Finally, we demonstrated the effectiveness of our framework in a defect spotting scenario where the operator supervises robot operations and modifies online the plan when necessary.

We explored rate mode control for virtual reality (VR) based robot teleoperation with constant and variable mapping of the human-operator’s joystick position to the speed (rate) of the robot’s end-effector. The variable mapping depended on the visual scale of the virtual reconstruction of the remote environment to the scale of the real remote environment. We demonstrated how the rate mode control and variable scaling based on the VR reconstruction scale can be efficiently used for seated VR based robot teleoperation when the operator’s arms are supported to reduce tiredness. The experimental study with five human participants demonstrated that variable mapping allowed participants to teleoperate the robot more effectively, by adjusting the VR visual scale albeit at a cost of increased perceived workload.

Abstract The manipulation of articulated objects is of primary importance in Robotics and can be considered as one of the most complex manipulation tasks. Traditionally, this problem has been tackled by developing ad hoc approaches, which lack flexibility and portability. In this paper, we present a framework based on answer set programming (ASP) for the automated manipulation of articulated objects in a robot control architecture. In particular, ASP is employed for representing the configuration of the articulated object for checking the consistency of such representation in the knowledge base and for generating the sequence of manipulation actions. The framework is exemplified and validated on the Baxter dual-arm manipulator in the first, simple scenario. Then, we extend such scenario to improve the overall setup accuracy and to introduce a few constraints in robot actions execution to enforce their feasibility. The extended scenario entails a high number of possible actions that can be fruitfully combined together. Therefore, we exploit macro actions from automated planning in order to provide more effective plans. We validate the overall framework in the extended scenario, thereby confirming the applicability of ASP also in more realistic Robotics settings and showing the usefulness of macro actions for the robot-based manipulation of articulated objects.

In this article, we propose FlexHRC+, a hierarchical human-robot cooperation architecture designed to provide collaborative robots with an extended degree of autonomy when supporting human operators in high-variability shop-floor tasks. The architecture encompasses three levels, namely for perception, representation, and action. Building up on previous work, here we focus on an in-the-loop decision-making process for the operations of collaborative robots coping with the variability of actions carried out by human operators, and the representation level, integrating a hierarchical and/or graph whose online behavior is formally specified using first-order logic. The architecture is accompanied by experiments including collaborative furniture assembly and object positioning tasks.

This paper addresses two intertwined needs for collaborative robots operating in shop-floor environments. The first is the ability to perform complex manipulation operations, such as those on articulated or even flexible objects, in a way robust to a high degree of variability in the actions possibly carried out by human operators during collaborative tasks. The second is encoding in such operations a basic knowledge about physical laws (e.g., gravity), and their effects on the models used by the robot to plan its actions, to generate more robust plans. We adopt the manipulation in three-dimensional space of articulated objects as an effective use case to ground both needs, and we use a variant of the Planning Domain Definition Language to integrate the planning process with a notion of gravity. Different complexity levels in modelling gravity are evaluated, which tradeoff model faithfulness and performance. A thorough validation of the framework is done in simulation using a dual-arm Baxter manipulator.

The presence and coexistence of human operators and collaborative robots in shop-floor environments raises the need for assigning tasks to either operators or robots, or both. Depending on task characteristics, operator capabilities and the involved robot functionalities, it is of the utmost importance to design strategies allowing for the concurrent and/or sequential allocation of tasks related to object manipulation and assembly. In this paper, we extend the FLEXHRC framework presented in [1] to allow a human operator to interact with multiple, heterogeneous robots at the same time in order to jointly carry out a given task. The extended FLEXHRC framework leverages a concurrent and sequential task representation framework to allocate tasks to either operators or robots as part of a dynamic collaboration process. In particular, we focus on a use case related to the inspection of product defects, which involves a human operator, a dual-arm Baxter manipulator from Rethink Robotics and a Kuka youBot mobile manipulator.

The problem of autonomous transportation in industrial scenarios is receiving a renewed interest due to the way it can revolutionise internal logistics, especially in unstructured environments. This paper presents a novel architecture allowing a robot to detect, localise, and track (possibly multiple) pallets using machine learning techniques based on an on-board 2D laser rangefinder only. The architecture is composed of two main components: the first stage is a pallet detector employing a Faster Region-Based Convolutional Neural Network (Faster R-CNN) detector cascaded with a CNN-based classifier; the second stage is a Kalman filter for localising and tracking detected pallets, which we also use to defer commitment to a pallet detected in the first stage until sufficient confidence has been acquired via a sequential data acquisition process. For fine-tuning the CNNs, the architecture has been systematically evaluated using a real-world dataset containing 340 labelled 2D scans, which have been made freely available in an online repository. Detection performance has been assessed on the basis of the average accuracy over k-fold cross-validation, and it scored 99.58% in our tests. Concerning pallet localisation and tracking, experiments have been performed in a scenario where the robot is approaching the pallet to fork. Although data have been originally acquired by considering only one pallet as per specification of the use case we consider, artificial data have been generated as well to mimic the presence of multiple pallets in the robot workspace. Our experimental results confirm that the system is capable of identifying, localising and tracking pallets with a high success rate while being robust to false positives.

This paper introduces a technique to teach robots how to represent and qualitatively interpret perceived scenes in tabletop scenarios. To this aim, we envisage a 3-step humanrobot interaction process, in which (i) a human shows a scene to a robot, (ii) the robot memories a symbolic scene representation (in terms of objects and their spatial arrangement), and (iii) the human can revise such a representation, if necessary, by further interacting with the robot; here, we focus on steps i and ii. Scene classification occurs at a symbolic level, using ontology-based instance checking and subsumption algorithms. Experiments showcase the main properties of the approach, i.e., detecting whether a new scene belongs to a scene class already represented by the robot, or otherwise creating a new representation with a one shot learning approach, and correlating scenes from a qualitative standpoint to detect similarities and differences in order to build a scene hierarchy.

The paper presents a software architecture allowing a robot to learn new compositions of objects in table-top scenarios by human demonstrations. The robot qualitatively represents those scenes, reason upon their similarity, and interact with humans through dialogues to talk about represented scenes. We formalise the robot behaviour based on a Description Logic representation of scenes through spatial beliefs, i.e., learned logic predicates, on which the robot applies symbolic reasoning to recognise and explain the scene. We exploit the logical structure of predicates in a software architecture that enables a robot exposing its beliefs, and if required, it allows a human supervisor to apply corrections in a form akin to robot active perception. The paper critically discusses the design of the software components and their interfaces, discriminating between knowledge representation and dialogue management. Those components are developed for human-robot knowledge sharing applications involving visual, verbal, and auditory modalities of interaction. Software components are treated as grey boxes managing an ontology-based formalisation of robot beliefs through four contextualised dialogues, for which we present a unique design pattern.

Gestures are a natural communication modality for humans. The ability to interpret gestures is fundamental for robots aiming to naturally interact with humans. Wearable sensors are promising to monitor human activity, in particular the usage of triaxial accelerometers for gesture recognition have been explored. Despite this, the state of the art presents lack of systems for reliable online gesture recognition using accelerometer data. The article proposes SLOTH, an architecture for online gesture recognition, based on a wearable triaxial accelerometer, a Recurrent Neural Network (RNN) probabilistic classifier and a procedure for continuous gesture detection, relying on modelling gesture probabilities, that guarantees (i) good recognition results in terms of precision and recall, (ii) immediate system reactivity.

Modern manufacturing paradigms introduce the need for robots able to naturally cooperate with humans in an unstructured and dynamic environment. In this article we extend FlexHRC, an architecture for flexible and collaborative manufacturing robots, with an online perception-simulation-planning framework that allows the robot to assess the status of the workspace, keeping track at all times of the stage at which the cooperative manufacturing process is, to identify its next action, to simulate it to check its feasibility and, as a consequence, to dynamically allocate tasks to itself or the human operator. We have tested the FlexHRC with a dual-arm manipulator cooperating with a person to assemble a table with one tabletop and four legs.

Cultural adaptation, i.e., the matching of a robot's behaviours to the cultural norms and preferences of its user, is a well known key requirement for the success of any assistive application. However, culture-dependent robot behaviours are often implicitly set by designers, thus not allowing for an easy and automatic adaptation to different cultures. This paper presents a method for the design of culture-aware robots, that can automatically adapt their behaviour to conform to a given culture. We propose a mapping from cultural factors to related parameters of robot behaviours which relies on linguistic variables to encode heterogeneous cultural factors in a uniform formalism, and on fuzzy rules to encode qualitative relations among multiple variables. We illustrate the approach in two practical case studies.

In this paper we introduce a gesture-based robot control framework, we discuss the adopted design principles and we report results about its evaluation with humans. Gesture-based control using wearable devices may constitute a novel form of human-robot interaction, but its implications have not been discussed in the literature. We discuss the main challenging issues, possible design guidelines and an open source, freely available implementation using commercially available devices and robots. The overall performance of the architecture, as well as its validation with 27 untrained volunteers, is reported.

In this paper, we present a novel cognitive framework allowing a robot to form memories of relevant traits of its perceptions and to recall them when necessary. The framework is based on two main principles: on the one hand, we propose an architecture inspired by current knowledge in human memory organisation; on the other hand, we integrate such an architecture with the notion of context, which is used to modulate the knowledge acquisition process when consolidating memories and forming new ones, as well as with the notion of familiarity, which is employed to retrieve proper memories given relevant cues. Although much research has been carried out, which exploits Machine Learning approaches to provide robots with internal models of their environment (including objects and occurring events therein), we argue that such approaches may not be the right direction to follow if a long-term, continuous knowledge acquisition is to be achieved. As a case study scenario, we focus on both robot–environment and human–robot interaction processes. In case of robot–environment interaction, a robot performs pick and place movements using the objects in the workspace, at the same time observing their displacement on a table in front of it, and progressively forms memories defined as relevant cues (e.g. colour, shape or relative position) in a context-aware fashion. As far as human–robot interaction is concerned, the robot can recall specific snapshots representing past events using both sensory information and contextual cues upon request by humans.

This paper proposes an integrated path following and obstacle avoidance framework for robots in crowded environments. The architecture considers two major requirements: (i) a given performance level for path following must be guaranteed; (ii) human-friendly robots must behave predictably and naturally, in order to avoid dangerous people reactions. In order to generate paths perceived as natural, and inspired by what happens in high traffic highways, we assume a path to consist of a family of curves, which can be switched using sensory information. The resulting behaviour is both predictable (paths are planned and validated beforehand) and natural (the overall qualitative behaviour is maintained when switching curves). The overall error is upper bounded by the curves configuration. The paper presents results in simulation.

In this paper, a robot learning approach is pro- posed which integrates Visuospatial Skill Learning, Imitation Learning, and conventional planning methods. In our approach, the sensorimotor skills (i.e., actions) are learned through a learning from demonstration strategy. The sequence of per- formed actions is learned through demonstrations using Visu- ospatial Skill Learning. A standard action-level planner is used to represent a symbolic description of the skill, which allows the system to represent the skill in a discrete, symbolic form. The Visuospatial Skill Learning module identifies the underlying constraints of the task and extracts symbolic predicates (i.e., action preconditions and effects), thereby updating the planner representation while the skills are being learned. Therefore the planner maintains a generalized representation of each skill as a reusable action, which can be planned and performed inde- pendently during the learning phase. Preliminary experimental results on the iCub robot are presented.

Human Odometry (HO) is the process of providing a person with a continuous estimate of their location, on the basis of information acquired solely by sensors carried around by the person themselves. In an effort towards the development of effective and robust HO systems, we present the Human Odometry Outdoor Dataset (HOOD), a public collection of labelled accelerometer and gyroscope data recordings. We compare four sensor placements (foot, waist, wrist, chest) to identify the most suitable placement for different types of motions (ranging from walking to slithering), occurring in highly diverse real environments (such as flat grass fields, staircases and rough terrains).

Robots for the elderly are a particular category of home assistive robots, helping people in the execution of daily life tasks to extend their independent life. Such robots should be able to determine the level of independence of the user and track its evolution over time, to adapt the assistance to the person capabilities and needs. Human Activity Recognition systems employ various sensing strategies, relying on environmental or wearable sensors, to recognize the daily life activities which provide insights on the health status of a person. The main contribution of the article is the design of an heterogeneous information management framework, allowing for the description of a wide variety of human activities in terms of multi-modal environmental and wearable sensing data and providing accurate knowledge about the user activity to any assistive robot.

We present a robot cognitive framework based on (a) a memory-like architecture; and (b) the notion of “context”. We posit that relying solely on machine learning techniques may not be the right approach for a long-term, continuous knowledge acquisition. Since we are interested in long-term human-robot interaction, we focus on a scenario where a robot “remembers” relevant events happening in the environment. By visually sensing its surroundings, the robot is expected to infer and remember snapshots of events, and recall specific past events based on inputs and contextual information from humans. Using a COTS vision frameworks for the experiment, we show that the robot is able to form “memories” and recall related events based on cues and the context given during the human-robot interaction process.

The papers in this special section present the most recent research work that showcases the state-of-the-art of human-centered computing and its potential applications in developing truly smart home automation systems.

Within the past decade, extensive research has been done on large-scale tactile sensing, as a result of which, a large variety of robot skins have been developed. These robot skins are different in various aspects: the sensing modality, interconnectivity of the sensors, modularity, the communication network, etc. This variety limits portability of software among these robot skins. In this article, a middleware is proposed that is capable of interacting in principle with any robot skin, through the use of simple drivers. Primarily, the middleware acquires data in real-time and provides its applications with those data in an abstract structure. As a result, the portability of algorithms implemented for large-scale tactile data processing is greatly increased among various available and future robot skins.

This article discusses a real-time networking infrastructure for a large-scale tactile sensing system to be used with humanoid robots. In such a system, real-time networking issues are of the utmost importance. Stemming from previous work, a theoretical model is presented and experimentally validated. Tests show real-time performance in a network of distributed computational nodes, each one in charge of managing part of the tactile system.

The automatic monitoring of specific Activities of Daily Living (ADL) can be a useful tool for Human-Robot Interaction in smart environments and Assistive Robotics applications. The qualitative definition that is given for most ADL and the lack of well-defined benchmarks, however, are obstacles toward the identification of the most effective monitoring approaches for different tasks. The contribution of the article is two-fold: (i) we propose a taxonomy of ADL allowing for their categorization with respect to the most suitable monitoring approach; (ii) we present a freely available dataset of acceleration data, coming from a wrist-worn wearable device, targeting the recognition of 14 different human activities.

The automatic assessment of the level of independence of a person, based on the recognition of a set of Activities of Daily Living, is among the most challenging research fields in Ambient Intelligence. The article proposes a framework for the recognition of motion primitives, relying on Gaussian Mixture Modeling and Gaussian Mixture Regression for the creation of activity models. A recognition procedure based on Dynamic Time Warping and Mahalanobis distance is found to: (i) ensure good classification results; (ii) exploit the properties of GMM and GMR modeling to allow for an easy run-time recognition; (iii) enhance the consistency of the recognition via the use of a classifier allowing unknown as an answer.

Tactile sensing is considered a key technology for implementing complex robot interaction tasks. The contribution of this article is two-fold: (i) we propose a general-purpose algorithm for the reconstruction of deformation and force distributions for capacitance-based skin-like systems; (ii) real-time performance can be tuned according to available computational resources, which leads to an any-time formulation. Experiments (both in simulation and with real robot skin) provide a quantitative analysis of results.

It is well known that there is a worldwide increase in both the number of elderly people and the number of elderly people with mild cognitive impairments [1], [2] and thus in need of assistance in the execution of activities of daily living. Socially Assistive Robotics is a novel research field, aiming at the design of robots relying on social means to interact with people and with a well-defined assistive purpose. The contribution of the article is three-fold: (i) a detailed analysis of the requirements of a socially assistive robot helping elderly people in the execution of everyday activities; (ii) the outline of the design principles for socially assistive robots; (iii) a first proposal for a wearable robot able to engage humans at the cognitive level.

In this paper, the description of a sensorized glove that has been developed to perform experiments in robot-based manipulation of clothes and objects is reported. The glove embeds a capacitive tactile sensing technology that has been designed in the past few years. The glove is used to provide an estimate of the expected tactile feedback related to involved forces and contact areas during common manipulation tasks. This information will be used in order to design a robot gripper for cloth manipulation.

This paper describes automated procedures for the design and deployment of artificial skin for humanoid robots. This problem is challenging under different perspectives: on the one hand, different robots are characterized by different shapes, thereby requiring a high degree of skin customization; on the other hand, it is necessary to define optimal criteria specifying how the skin must be placed on robot parts. This paper addresses the problem of optimally covering robot parts with tactile sensors, discussing possible solutions with reference to a specific artificial skin technology for robots, which has been developed in the past few years. Results show that it is possible to automate the majority of the required steps, with promising results in view of a future complete automation of the process.

This paper introduces semantic-aware real-time (SeART), an extension to conventional operating systems, which deals with complex real-time robotics applications. SeART addresses the problem of selecting a subset of tasks to be scheduled depending on the current operating context: mission objectives, other tasks currently executed, the availability or unavailability of sensors and other resources, as well as temporal constraints. Toward this end, SeART is able to represent the semantics of tasks to be scheduled, i.e., what tasks are meant for, and to use this information in the scheduling process. This paper describes in detail the SeART architecture by focusing on representations and reasoning procedures, and presenting a case-study which is related to mobile robotics for autonomous objects transportation.

The problem of "providing robots with the sense of touch" is fundamental in order to develop the next generations of robots capable of interacting with humans in different contexts: in daily housekeeping activities, as working partners or as caregivers, just to name a few.

This article introduces an extension of the original Parallel Force-Position control framework based on the use of tactile maps. Whole body skin systems for humanoid robots are considered a fundamental feature to improve contact interaction tasks by means of large scale tactile feedback. Stemming from previous work [1], the article describes how it is possible to extend the Parallel Force-Position control paradigm with the use of tactile maps, i.e., computational representation structures encoding the position of tactile elements located on the robot body surface. Furthermore, tactile maps are used to encode specific local features of contact trajectories, such as the desired exerted force at the contact point and the desired velocity. Results in simulation validate the approach.

Humanoid robots operating in the real world must exhibit very complex behaviors, such as object manipulation or interaction with people. Such capabilities pose the problem of being able to reason on a huge number of different objects, places and actions to carry out, each one relevant for achieving robot goals. This article proposes a functional representation of objects, places and actions described in terms of affordances and capabilities. Everyday problems can be efficiently dealt with by decomposing the reasoning process in two phases, namely problem awareness (which is the focus of this article) and action selection.

The article describes a tool for the fast prototyping and deployment of context-aware applications, in particular to welcome visitors in urban areas. The system has been conceived to guarantee continuous access to everybody, everywhere, at any time, and therefore it does not rely on any special device to connect visitors to the intelligent environment. In fact, we assume that visitors are equipped with low-end mobile phones embedded with bluetooth technology, which provide approximate positioning information. In spite of this, the system must be able to assess the current context in terms of user location, preferences, current activity, and to suggest city-tours and activities which meets the most the visitor's expectations. The article shows how, basing on Google maps API and OWLDL ontologies, the rapid prototyping and rapid deployment of outdoor context-aware applications based on bluetooth messaging and positioning information can be achieved.

This paper deals with the problem of estimating the position of tactile elements (i.e. taxels) that are mounted on a robot body part. This problem arises with the adoption of tactile systems with a large number of sensors, and it is particularly critical in those cases in which the system is made of flexible material that is deployed on a curved surface. In this scenario the location of each taxel is partially unknown and difficult to determine manually. Placing the device is in fact an inaccurate procedure that is affected by displacements in both position and orientation. Our approach is based on the idea that it is possible to automatically infer the position of the taxels by measuring the interaction forces exchanged between the sensorized part and the environment. The location of the contact is estimated through force/torque (F/T) measures gathered by a sensor mounted on the kinematic chain of the robot. Our method requires few hypotheses and can be effectively implemented on a real platform, as demonstrated by the experiments with the iCub humanoid robot.

Summary form only given. Scaled technologies continue to exhibit variability, driven by both random process effects and systematic structural effects. Process and design rule actions can be taken to reduce, or even eliminate, sources of systematic variability. Random variability is more difficult to combat, but architectural decisions can be made to limit the device sensitivity to specific random effects. A review of several current sources of technology variability is presented, and the impacts to the overall technology offering are assessed.

This article introduces a model for representing motion primitives for entertainment humanoid robots using Generalized Hierarchical AND/OR graphs. On the one hand, the goal is to drive robots with scripts, as if they were on a stage. On the other hand, the approach allows for storing a minimum amount of behaviours, thereby reducing on-board memory and computational requirements. Standard ontology-based reasoning mechanisms are used to operate on such a representation, in a fully hierarchical fashion. Experimental validation has been assessed using toy Kondo robots.

This paper deals with the problem of calibrating a large number of tactile elements (i.e., taxels) organized in a skin sensor system after fixing them to a robot body part. This problem has not received much attention in literature because of the lack of large-scale skin sensor systems. The proposed approach is based on a controlled compliance motion with respect to external objects whose pose is known, which allows a robot to determine the location of its own taxels. The major contribution of this work is the formulation of the skin calibration problem as a maximum-likelihood mapping problem in a 6D space, where both the position and the orientation of each taxel are recovered. An effective calibration process is envisaged that, given a compliance control law that assures prolonged contact maintenance between a given body part and an external object, returns a maximum-likelihood estimate of detected taxel poses. Simulations validate the approach.

The focus of the present work is on daily activity planning, i.e., representations and algorithms able to produce a course of action to deal efficiently with problems of daily living. To achieve this, the article proposes a functional representation of everyday objects, places and actions described in terms of affordances. Its contributions are two--fold: (i) it proposes to represent affordances and capabilities as regions in a proper affordance and capability space, and to describe such regions using neural maps; (ii) it proposes to decompose the planning process into different activities, by introducing a phase referred to as Problem Awareness preceding Action Planning, which allows to reduce the planning space in order to tackle large-scale planning problems.

In this paper a framework for representing tactile information in robots is discussed. Control models exploiting tactile sensing are fundamental in social Human-Robot interaction tasks. Difficulties arising in rendering the sense of touch in robots are at different levels: both representation and computational issues must be considered. A layered system is proposed, which is inspired from tactile sensing in humans for building artificial somatosensory maps in robots. Experiments in simulation are used to validate the approach.

In this paper a framework for representing tactile information in robots is discussed. Control models exploiting tactile sensing are fundamental in social Human-Robot interaction tasks. Difficulties arising in rendering the sense of touch in robots are at different levels: both representation and computational issues must be considered. A layered system is proposed, which is inspired from tactile sensing in humans for building artificial somatosensory maps in robots. Experiments in simulation are used to validate the approach.

This paper describes a framework for representing physical interaction tasks using tactile feedback. Although contact feedback has been widely exploited to control interaction with objects, the direct use of tactile information in designing and representing physical interaction rules has not received comparable attention in the literature. The missing link between algorithms implementing models of interaction and frameworks providing tactile information is one of the possible reasons. The major contribution of the paper is a working method to build a map of tactile sensors attached to a robot body and a control law using such a map to tune physical interaction with an external object. Experiments are used to validate the approach.

This paper presents an architecture for context-aware Ubiquitous Robotics applications, where mobile robots cooperate with intelligent environments to fulfill their tasks. Specifically, the work is focused on distributed knowledge representation issues and context assessment strategies, and introduces a technique for on-line context recognition in highly dynamic environments. Experimental validation, performed in a civillian hospital building, is described and discussed.

A knowledge representation system is introduced that allows the recognition of temporal patterns of events in context-aware environments. The system is based on standard frameworks, such as an ontology, an inference mechanism, and relational operators acting on numerical quantities. The paper describes how knowledge is managed, then introduces a collection of temporal operators that are inspired by the Allen's interval algebra, and then details a situation recognition algorithm to assess knowledge semantics. An example is reported to describe the approach.

This paper presents muNav, a novel approach to navigation which, with minimal requirements in terms of onboard sensory, memory, and computational power, exhibits way-finding behaviors in very complex environments. The algorithm is intrinsically robust, since it does not require any internal geometrical representation or self-localization capabilities. Experimental results, performed with both simulated and real robots, validate the proposed theoretical approach.

A framework is introduced that is aimed at integrating ontology and logic approaches for context-awareness, suitable for use in Ambient Intelligence (AmI) scenarios. In particular, the context description language CDL is described, which allows to easily specify patterns of events which occurrences must be monitored by actual systems. As long as systems evolve, symbolic data originating from heterogeneous sources are first aggregated and then classified according to formulas described in CDL. Experimental results performed in a Smart Home environment are presented and discussed.

This paper describes ANSER, a system designed to perform surveillance in civilian airports and similar wide outdoor areas. Whereas an intelligent system - possibly controlled by a human supervisor - is able to integrate the information originating from different sources (i.e., fixed devices and sensors distributed throughout the environment) and to coordinate their behaviors in case of anomalies, the mobile robot is a significant part of the overall system: its main subsystems, i.e., autonomous surveillance, localization (performed using only a non-differential GPS and a laser rangefinder) and navigation are investigated in depth. Experimental results validate the robustness and reliability of the approach.

The paper deals with the role of line features in mobile robot self-localization, when an extended Kalman filter is adopted for position tracking. First, a theoretical analysis is introduced, showing how the "length" of each extracted line (i.e., the number of the contributing range measurements) affects the localization accuracy. Second, a novel approach that takes into account the main findings of the theoretical analysis is considered. Finally, experimental results are used to validate the system.