Cylinder-specific condition monitoring is critical for ensuring reliability and safety in large combustion engines. However, conventional approaches typically require extensive instrumentation, resulting in high implementation costs and complexity. This paper presents a data-driven and sensor-efficient condition monitoring methodology for large industrial engines that achieves accurate fault detection using only a single flywheel encoder measurement. Despite the scarcity of real-world fault data, the framework leverages simulation-based training enhanced by domain randomization, feature alignment, and semi-supervised learning techniques to bridge the simulation-to-reality gap. A modified Deep Convolutional Neural Network with Wide First-layer Kernels (WDCNN) is employed for robust fault classification. The framework is validated on a 20-cylinder gas engine. Compared to conventional lateral vibration- or pressure-based monitoring systems that rely on distributed multi-sensor frameworks, this approach achieves comparable accuracy with drastically reduced sensor requirements, reducing the required amount from up to 40 to 1. A lower number of required sensors maintain higher reliability levels, since higher sensor counts expose the system to a higher rate of sensor failure. Experimental results show 100% fault detection accuracy and 95.7% classification accuracy on a dataset consisting of limited real measured data, highlighting the framework’s potential for practical deployment in real-world industrial settings with minimal sensor setups. Validation was conducted using a single real-world fault condition, underscoring the need for future validation on broader fault generalization. Nevertheless, this work demonstrates the feasibility of high-precision engine fault monitoring with dramatically reduced sensor requirements, enabling cost-effective diagnostics in data-scarce industrial environments.

In the last decade, we have witnessed the introduction of several novel deep neural network (DNN) architectures exhibiting ever-increasing performance across diverse tasks. Explaining the upward trend of their performance, however, remains difficult as different DNN architectures of comparable depth and width – common factors associated with their expressive power – may exhibit a drastically different performance even when trained on the same dataset. In this paper, we introduce the concept of the non-linearity signature of DNN, the first theoretically sound solution for approximately measuring the non-linearity of deep neural networks. Built upon a score derived from closed-form optimal transport mappings, this signature provides a better understanding of the inner workings of a wide range of DNN architectures and learning paradigms, with a particular emphasis on the computer vision task. We provide extensive experimental results that highlight the practical usefulness of the proposed non-linearity signature and its potential for long-reaching implications. The code for our work is available at https://github.com/qbouniot/AffScoreDeep.

Simulation of conflict situations for autonomous driving research is crucial for understanding and managing interactions between Automated Vehicles (AVs) and human drivers. This paper presents a set of exemplary conflict scenarios in CARLA that arise in shared autonomy settings, where both AVs and human drivers must navigate complex traffic environments. We explore various conflict situations, focusing on the impact of driver behavior and decision-making processes on overall traffic safety and efficiency. We build a simple extendable toolkit for situation awareness research, in which the implemented conflicts can be demonstrated.

Learning to perform accurate and rich simulations of human driving behaviors from data for autonomous vehicle testing remains challenging due to human driving styles' high diversity and variance. We address this challenge by proposing a novel approach that leverages contrastive learning to extract a dictionary of driving styles from pre-existing human driving data. We discretize these styles with quantization, and the styles are used to learn a conditional diffusion policy for simulating human drivers. Our empirical evaluation confirms that the behaviors generated by our approach are both safer and more human-like than those of the machine-learning-based baseline methods. We believe this has the potential to enable higher realism and more effective techniques for evaluating and improving the performance of autonomous vehicles.

Multi Robot Systems are increasingly deployed in applications, such as intralogistics or autonomous delivery, where multiple robots collaborate to complete tasks efficiently. One of the key factors enabling their efficient cooperation is Multi-Robot Task Allocation (MRTA). Algorithms solving this problem optimize task distribution among robots to minimize the overall execution time. In shared environments, apart from the relative distance between the robots and the tasks, the execution time is also significantly impacted by the delay caused by navigating around moving people. However, most existing MRTA approaches are dynamics-agnostic, relying on static maps and neglecting human motion patterns, leading to inefficiencies and delays. In this paper, we introduce Human-Aware Task Allocation (HATA). This method leverages Maps of Dynamics (MoDs), spatio-temporal queryable models designed to capture historical human movement patterns, to estimate the impact of humans on the task execution time during deployment. HATA utilizes a stochastic cost function that includes MoDs Experimental results show that integrating MoDs enhances task allocation performance, resulting in reduced mission completion times by up to 26% compared to the dynamics-agnostic method and up to 19% compared to the baseline. This work underscores the importance of considering human dynamics in MRTA within shared environments and presents an efficient framework for deploying multi-robot systems in environments populated by humans.

The task of co-optimizing the body and behaviour of agents has been a long-standing problem in the fields of evolutionary robotics and embodied AI. Previous work has largely focused on the development of learning methods exploiting massive parallelization of agent evaluations with large population sizes, a paradigm which is applicable to simulated agents but cannot be transferred to the real world due to the assoicated costs with the production of embodiments and robots. Furthermore, recent data-efficient approaches utilizing reinforcement learning can suffer from distributional shifts in transition dynamics as well as in state and action spaces when experiencing new body morphologies. In this work, we propose a new co-adaptation method combining reinforcement learning and State-Aligned Self-Imitation Learning to co-design embodiment and behavioural policies withing a handful of design iterations. We show that the integration of a self-imitation signal improves the data-efficiency of the co-adaptation process as well as the behavioural recovery when adapting morphological parameters.

Modern-day autonomous robots need high-level map representations to perform sophisticated tasks. Recently, 3D scene graphs (3DSGs) have emerged as a promising alternative to traditional grid maps, blending efficient memory use and rich feature representation. However, most efforts to apply them have been limited to static worlds. This work introduces REACT, a framework that efficiently performs real-time attribute clustering and transfer to relocalize object nodes in a 3DSG. REACT employs a novel method for comparing object instances using an embedding model trained on triplet loss, facilitating instance clustering and matching. Experimental results demonstrate that REACT is able to relocalize objects while maintaining computational efficiency. The REACT framework’s source code will be available as an open-source project, promoting further advancements in reusable and up-datable 3DSGs1.

Visual-language models (VLMs) have recently been introduced in robotic mapping using the latent representations, i.e., embeddings, of the VLMs to represent semantics in the map. They allow moving from a limited set of human-created labels toward open-vocabulary scene understanding, which is very useful for robots when operating in complex real-world environments and interacting with humans. While there is anecdotal evidence that maps built this way support downstream tasks, such as navigation, rigorous analysis of the quality of the maps using these embeddings is missing. In this paper, we propose a way to analyze the quality of maps created using VLMs. We investigate two critical properties of map quality: queryability and distinctness. The evaluation of queryability addresses the ability to retrieve information from the embeddings. We investigate intra-map distinctness to study the ability of the embeddings to represent abstract semantic classes and inter-map distinctness to evaluate the generalization properties of the representation. We propose metrics to evaluate these properties and evaluate two state-of-the-art mapping methods, VLMaps and OpenScene, using two encoders, LSeg and OpenSeg, using real-world data from the Matterport3D data set. Our findings show that while 3D features improve queryability, they are not scale invariant, whereas image-based embeddings generalize to multiple map resolutions. This allows the image-based methods to maintain smaller map sizes, which can be crucial for using these methods in real-world deployments. Furthermore, we show that the choice of the encoder has an effect on the results. The results imply that properly thresholding open-vocabulary queries is an open problem.

Despite the potential the ability to identify granular materials creates for applications such as robotic cooking or earthmoving, granular material identification remains a challenging area, existing methods mostly relying on shaking the materials in closed containers. This work presents an interactive material identification framework that enables robots to identify a wide range of granular materials using only force-torque measurements. Unlike prior works, the proposed approach uses direct interaction with the materials. The approach is evaluated through experiments with a real-world dataset comprising 11 granular materials, which we also make publicly available. Results show that our method can identify a wide range of granular materials with near-perfect accuracy while relying solely on force measurements obtained from direct interaction. Further, our comprehensive data analysis and experiments show that a high-performancefeature space must combine features related to the force signal’s time-domain dynamics and frequency spectrum. We account for this by proposing a combination of the raw signal and its high-frequency magnitude histogram as the suggested feature space representation. We show that the proposed feature space outperforms baselines by a significant margin. The code and data set are available at: https://irobotics.aalto.fi/identify_granular/.

Learning from Demonstration (LfD) allows robots to mimic human actions. However, these methods do not model constraints crucial to ensure safety of the learned skill. Moreover, even when explicitly modelling constraints, they rely on the assumption of a known cost function, which limits their practical usability for task with unknown cost. In this work we propose a two-step optimization process that allow to estimate cost and constraints by decoupling the learning of cost functions from the identification of unknown constraints within the demonstrated trajectories. Initially, we identify the cost function by isolating the effect of constraints on parts of the demonstrations. Subsequently, a constraint leaning method is used to identify the unknown constraints. Our approach is validated both on simulated trajectories and a real robotic manipulation task. Our experiments show the impact that incorrect cost estimation has on the learned constraints and illustrate how the proposed method is able to infer unknown constraints, such as obstacles, from demonstrated trajectories without any initial knowledge of the cost.

Imitation Learning (IL) is a promising paradigm for learning dynamic manipulation of deformable objects since it does not depend on difficult-to-create accurate simulations of such objects. However, the translation of motions demonstrated by a human to a robot is a challenge for IL, due to differences in the embodiments and the robot’s physical limits. These limits are especially relevant in dynamic manipulation where high velocities and accelerations are typical. To address this problem, we propose a framework that first maps a dynamic demonstration into a motion that respects the robot’s constraints using a constrained Dynamic Movement Primitive. Second, the resulting object state is further optimized by quasi-static refinement motions to optimize task performance metrics. This allows both efficiently altering the object state by dynamic motions and stable small-scale refinements. We evaluate the framework in the challenging task of bag opening, designing the system BILBO: Bimanual dynamic manipulation using Imitation Learning for Bag Opening. Our results show that BILBO can successfully open a wide range of crumpled bags, using a demonstration with a single bag. See supplementary material at https://sites.google.com/view/bilbo-bag.

This work presents a framework for automatically extracting physical object properties, such as material composition, mass, volume, and stiffness, through robot manipulation and a database of object measurements. The framework involves exploratory action selection to maximize learning about objects on a table. A Bayesian network models conditional dependencies between object properties, incorporating prior probability distributions and uncertainty associated with measurement actions. The algorithm selects optimal exploratory actions based on expected information gain and updates object properties through Bayesian inference. Experimental evaluation demonstrates effective action selection compared to a baseline and correct termination of the experiments if there is nothing more to be learned. The algorithm proved to behave intelligently when presented with trick objects with material properties in conflict with their appearance. The robot pipeline integrates with a logging module and an online database of objects, containing over 24,000 measurements of 63 objects with different grippers. All code and data are publicly available, facilitating automatic digitization of objects and their physical properties through exploratory manipulations.

Mapping people dynamics is a crucial skill for robots, because it enables them to coexist in human-inhabited environments. However, learning a model of people dynamics is a time consuming process which requires observation of large amount of people moving in an environment. Moreover, approaches for mapping dynamics are unable to transfer the learned models across environments: each model is only able to describe the dynamics of the environment it has been built in. However, the impact of architectural geometry on people’s movement can be used to anticipate their patterns of dynamics, and recent work has looked into learning maps of dynamics from occupancy. So far however, approaches based on trajectories and those based on geometry have not been combined. In this work we propose a novel Bayesian approach to learn people dynamics able to combine knowledge about the environment geometry with observations from human trajectories. An occupancy-based deep prior is used to build an initial transition model without requiring any observations of pedestrian; the model is then updated when observations become available using Bayesian inference. We demonstrate the ability of our model to increase data efficiency and to generalize across real large-scale environments, which is unprecedented for maps of dynamics.

We present Robot-centric Pooling (RcP), a novel pooling method designed to enhance end-to-end visuomo-tor policies by enabling differentiation between the robots and similar entities or their surroundings. Given an image-proprioception pair, RcP guides the aggregation of image features by highlighting image regions correlating with the robot’s proprioceptive states, thereby extracting robot-centric image representations for policy learning. Leveraging contrastive learning techniques, RcP integrates seamlessly with existing visuomotor policy learning frameworks and is trained jointly with the policy using the same dataset, requiring no extra data collection involving self-distractors. We evaluate the proposed method with reaching tasks in both simulated and real-world settings. The results demonstrate that RcP significantly enhances the policies’ robustness against various unseen distractors, including self-distractors, positioned at different locations. Additionally, the inherent robot-centric characteristic of RcP enables the learnt policy to be far more resilient to aggressive pixel shifts compared to the baselines. Code available at: https://github.com/Zheyu-Zhuang/RcP

Learning contextual and spatial environmental representations enhances autonomous vehicle’s hazard anticipation and decision-making in complex scenarios. Recent perception systems enhance spatial understanding with sensor fusion but often lack global environmental context. Humans, when driving, naturally employ neural maps that integrate various factors such as historical data, situational subtleties, and behavioral predictions of other road users to form a rich contextual understanding of their surroundings. This neural map-based comprehension is integral to making informed decisions on the road. In contrast, even with their significant advancements, autonomous systems have yet to fully harness this depth of human-like contextual understanding. Motivated by this, our work draws inspiration from human driving patterns and seeks to formalize the sensor fusion approach within an end-to-end autonomous driving framework. We introduce a framework that integrates three cameras (left, right, and center) to emulate the human field of view, coupled with top-down bird-eye-view semantic data to enhance contextual representation. The sensor data is fused and encoded using a self-attention mechanism, leading to an auto-regressive waypoint prediction module. We treat feature representation as a sequential problem, employing a vision transformer to distill the contextual interplay between sensor modalities. The efficacy of the proposed method is experimentally evaluated in both open and closed-loop settings. Our method achieves displacement error by 0 . 67 m in open-loop settings, surpassing current methods by 6.9% on the nuScenes dataset. In closed-loop evaluations on CARLA’s Town05 Long and Longest6 benchmarks, the proposed method enhances driving performance, route completion, and reduces infractions.

The co-adaptation of robots has been a long-standing research endeavour with the goal of adapting both body and behaviour of a robot for a given task, inspired by the natural evolution of animals. Co-adaptation has the potential to eliminate costly manual hardware engineering as well as improve the performance of systems. The standard approach to co-adaptation is to use a reward function for optimizing behaviour and morphology. However, defining and constructing such reward functions is notoriously difficult and often a significant engineering effort. This paper introduces a new viewpoint on the co-adaptation problem, which we call co-imitation: finding a morphology and a policy that allow an imitator to closely match the behaviour of a demonstrator. To this end we propose a co-imitation methodology for adapting behaviour and morphology by matching state-distributions of the demonstrator. Specifically, we focus on the challenging scenario with mismatched state- and action-spaces between both agents. We find that co-imitation increases behaviour similarity across a variety of tasks and settings, and demonstrate co-imitation by transferring human walking, jogging and kicking skills onto a simulated humanoid.

Planning robotic manipulation tasks, especially those that involve interaction between deformable and rigid objects, is challenging due to the complexity in predicting such interactions. We introduce SPONGE, a sequence planning pipeline powered by a deep learning-based contact prediction model for contacts between deformable and rigid bodies under interactions. The contact prediction model is trained on synthetic data generated by a developed simulation environ-ment to learn the mapping from point-cloud observation of a rigid target object and the pose of a deformable tool, to 3D representation of the contact points between the two bodies. We experimentally evaluated the proposed approach for a dish cleaning task both in simulation and on a real Franka Emika Panda with real-world objects. The experimental results demonstrate that in both scenarios the proposed planning pipeline is capable of generating high-quality trajectories that can accomplish the task by achieving more than 90% area coverage on different objects of varying sizes and curvatures while minimizing travel distance. Code and video are available at: https://irobotics.aalto.fi/sponge/.

Most state-of-the-art data-driven grasp sampling methods propose stable and collision-free grasps uniformly on the target object. For bin-picking, executing any of those reachable grasps is sufficient. However, for completing specific tasks, such as squeezing out liquid from a bottle, we want the grasp to be on a specific part of the object's body while avoiding other locations, such as the cap. This work presents a generative grasp sampling network, VCGS, capable of constrained 6-Degrees of Freedom (DoF) grasp sampling. In addition, we also curate a new dataset designed to train and evaluate methods for constrained grasping. The new dataset, called CONG, consists of over 14 million training samples of synthetically rendered point clouds and grasps at random target areas on 2889 objects. VCGS is benchmarked against GraspNet, a state-of-the-art unconstrained grasp sampler, in simulation and on a real robot. The results demonstrate that VCGS achieves a 10-15% higher grasp success rate than the baseline while being 2–3 times as sample efficient. Supplementary material is available on our project website.

Pre-defined manipulation primitives are widely used for cloth manipulation. However, cloth properties such as its stiffness or density can highly impact the performance of these primitives. Although existing solutions have tackled the parameterisation of pick and place locations, the effect of factors such as the velocity or trajectory of quasi-static and dynamic manipulation primitives has been neglected. Choosing appropriate values for these parameters is crucial to cope with the range of materials present in house-hold cloth objects. To address this challenge, we introduce the Quasi-Dynamic Parameterisable (QDP) method, which optimises parameters such as the motion velocity in addition to the pick and place positions of quasi-static and dynamic manipulation primitives. In this work, we leverage the framework of Sequential Reinforcement Learning to decouple sequentially the parameters that compose the primitives. To evaluate the effectiveness of the method, we focus on the task of cloth unfolding with a robotic arm in simulation and real-world experiments. Our results in simulation show that by deciding the optimal parameters for the primitives the performance can improve by 20% compared to sub-optimal ones. Real-world results demonstrate the advantage of modifying the velocity and height of manipulation primitives for cloths with different mass, stiffness, shape, and size. Supplementary material, videos, and code, can be found at https://sites.google.com/view/qdp-srl.

Learning policies from multiple demonstrators is often difficult because different individuals perform the same task differently due to hidden factors such as preferences. In the context of policy learning, this leads to multimodal policies. Existing policy learning methods often converge to a single solution mode, failing to capture the diversity in the solution space. In this paper, we introduce an imitation-guided reinforcement learning framework to solve the multimodal policy learning problem from a limited number of state-only demonstrations. Then, we propose LfBD (Learning from Behaviourally diverse Demonstration), an algorithm that builds a parameterised solution space to capture the variability in the behaviour space defined by demonstrations. To this end, we define a projection function based on the state density distributions from demonstrations to define such space. Our goal is not only to learn how to solve the task as the human demonstrator but also to extrapolate beyond the provided demonstrations. In addition, we show that with our method, we can perform a post-hoc policy search in the built solution space to recover policies that satisfy specific constraints or to find a policy that matches a given (state-only) behaviour.

Abstract The slowness principle is a concept inspired by the visual cortex of the brain. It postulates that the underlying generative factors of a quickly varying sensory signal change on a different, slower time scale. By applying this principle to state-of-the-art unsupervised representation learning methods one can learn a latent embedding to perform supervised downstream regression tasks more data efficient. In this paper, we compare different approaches tounsupervised slow representation learningsuch as $$L_p$$ Lp norm based slowness regularization and the SlowVAE, and propose a new term based on Brownian motion used in our method, the S-VAE. We empirically evaluate these slowness regularization terms with respect to their downstream task performance and data efficiency in state estimation and behavioral cloning tasks. We find that slow representations show great performance improvements in settings where only sparse labeled training data is available. Furthermore, we present a theoretical and empirical comparison of the discussed slowness regularization terms. Finally, we discuss how the Fréchet Inception Distance (FID), commonly used to determine the generative capabilities of GANs, can predict the performance of trained models in supervised downstream tasks.

Manipulating unknown objects in a cluttered environment is difficult because segmentation of the scene into objects, that is, object composition, is uncertain. Due to the uncertainty, prior work has either identified the “best” object composition and decided on manipulation actions accordingly or tried to greedily gather information about the “best” object composition. We instead, first, use different possible object compositions in planning, second, utilize object composition information provided by robot actions, third, consider the effect of competing object hypotheses on the desired task. We cast the manipulation planning problem as a partially observable Markov decision process (POMDP) that plans over possible object composition hypotheses. The POMDP chooses the action that maximizes long-term expected task-specific utility, and while doing so, considers informative actions and the effect of different object hypotheses on succeeding in the task. In simulation and physical robotic experiments, a probabilistic approach outperforms using the most likely object composition, and long term planning outperforms greedy decision making.

Robotic manipulation of cloth is a challenging task due to the high dimensionality of the configuration space and the complexity of dynamics affected by various material properties. The effect of complex dynamics is even more pronounced in dynamic folding, for example, when a square piece of fabric is folded in two by a single manipulator. To account for the complexity and uncertainties, feedback of the cloth state using e.g. vision is typically needed. However, construction of visual feedback policies for dynamic cloth folding is an open problem. In this paper, we present a solution that learns policies in simulation using Reinforcement Learning (RL) and transfers the learned policies directly to the real world. In addition, to learn a single policy that manipulates multiple materials, we randomize the material properties in simulation. We evaluate the contributions of visual feedback and material randomization in real-world experiments. The experimental results demonstrate that the proposed solution can fold successfully different fabric types using dynamic manipulation in the real world. Code, data, and videos are available at https://sites.google.com/view/dynamic-cloth-folding.

Evaluation of grasps on deformable$3\mathrm{D}$objects is a little-studied problem, even if the applicability of rigid object grasp quality measures for deformable ones is an open question. A central issue with most quality measures is their dependence on contact points, which for deformable objects depend on the deformations. This paper proposes a grasp quality measure for deformable objects that uses information about object deformation to calculate the grasp quality. Grasps are evaluated by simulating the deformations during grasping and predicting the contacts between the gripper and the grasped object. The contact information is then used as input for a new grasp quality metric to quantify the grasp quality. The approach is benchmarked against two classical rigid-body quality metrics on over 600 grasps in the Isaac gym simulation and over 50 real-world grasps. Experimental results show an average improvement of 18% in the grasp success rate for deformable objects compared to the classical rigid-body quality metrics. Furthermore, the proposed approach is approximately fifteen times faster to calculate than the shake task, which, to date, is one of the most reliable approaches to quantify a grasp on a deformable object.

High-Definition (HD) maps are needed for robust navigation of autonomous vehicles, limited by the on-board storage capacity. To solve this, we propose a novel framework, Environment-Aware Normal Distributions Transform (EA-NDT), that significantly improves compression of standard NDT map representation. The compressed representation of EA-NDT is based on semantic-aided clustering of point clouds resulting in more optimal cells compared to grid cells of standard NDT. To evaluate EA-NDT, we present an open-source implementation that extracts planar and cylindrical primitive features from a point cloud and further divides them into smaller cells to represent the data as an EA-NDT HD map. We collected an open suburban environment dataset and evaluated EA-NDT HD map representation against the standard NDT representation. Compared to the standard NDT, EA-NDT achieved consistently at least 1.5× higher map compression while maintaining the same descriptive capability. Moreover, we showed that EA-NDT is capable of producing maps with significantly higher descriptivity score when using the same number of cells than the standard NDT.

Driving in a complex and dynamic urban environment is a difficult task that requires a complex decision policy. In order to make informed decisions, one needs to gain an understanding of the long-range context and the importance of other vehicles. In this work, we propose to use Vision Transformer (ViT) to learn a driving policy in urban settings with birds-eye-view (BEV) input images. The ViT network learns the global context of the scene more effectively than with earlier proposed Convolutional Neural Networks (ConvNets). Furthermore, ViT’s attention mechanism helps to learn an attention map for the scene which allows the ego car to determine which surrounding cars are important to its next decision. We demonstrate that a DQN agent with a ViT backbone outperforms baseline algorithms with ConvNet backbones pre-trained in various ways. In particular, the proposed method helps reinforcement learning algorithms to learn faster, with increased performance and less data than baselines.

While 2D occupancy maps commonly used in mobile robotics enable safe navigation in indoor environments, in order for robots to understand and interact with their environment and its inhabitants representing 3D geometry and semantic environment information is required. Semantic information is crucial in effective interpretation of the meanings humans attribute to different parts of a space, while 3D geometry is important for safety and high-level understanding. We propose a pipeline that can generate a multi-layer representation of indoor environments for robotic applications. The proposed representation includes 3D metric-semantic layers, a 2D occupancy layer, and an object instance layer where known objects are replaced with an approximate model obtained through a novel model-matching approach. The metric-semantic layer and the object instance layer are combined to form an augmented representation of the environment. Experiments show that the proposed shape matching method outperforms a state-of-the-art deep learning method when tasked to complete unseen parts of objects in the scene. The pipeline performance translates well from simulation to real world as shown by F1-score analysis, with semantic segmentation accuracy using Mask R-CNN acting as the major bottleneck. Finally, we also demonstrate on a real robotic platform how the multi-layer map can be used to improve navigation safety.

We present a data-efficient framework for solving sequential decision-making problems which exploits the combination of reinforcement learning (RL) and latent variable generative models. The framework, called GenRL, trains deep policies by introducing an action latent variable such that the feed-forward policy search can be divided into two parts: (i) training a sub-policy that outputs a distribution over the action latent variable given a state of the system, and (ii) unsupervised training of a generative model that outputs a sequence of motor actions conditioned on the latent action variable. GenRL enables safe exploration and alleviates the data-inefficiency problem as it exploits prior knowledge about valid sequences of motor actions. Moreover, we provide a set of measures for evaluation of generative models such that we are able to predict the performance of the RL policy training prior to the actual training on a physical robot. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training on two robotics tasks: shooting a hockey puck and throwing a basketball. Furthermore, we empirically demonstrate that GenRL is the only method which can safely and efficiently solve the robotics tasks compared to two state-of-the-art RL methods.

Design teams of social robots are often multidisciplinary, due to the broad knowledge from different scientific domains needed to develop such complex technology. However, tools to facilitate multidisciplinary collaboration are scarce. We introduce a framework for the participatory design of social robots and corresponding canvas tool for participatory design. The canvases can be applied in different parts of the design process to facilitate collaboration between experts of different fields, as well as to incorporate prospective users of the robot into the design process. We investigate the usability of the proposed canvases with two social robot design case studies: a robot that played games online with teenage users and a librarian robot that guided users at a public library. We observe through participants’ feedback that the canvases have the advantages of (1) providing structure, clarity, and a clear process to the design; (2) encouraging designers and users to share their viewpoints to progress toward a shared one; and (3) providing an educational and enjoyable design experience for the teams.

While there exists many methods for manipulating rigid objects with parallel-jaw grippers, grasping with multi-finger robotic hands remains a quite unexplored research topic. Reasoning and planning collision-free trajectories on the additional degrees of freedom of several fingers represents an important challenge that, so far, involves computationally costly and slow processes. In this work, we present Multi-FinGAN, a fast generative multi-finger grasp sampling method that synthesizes high quality grasps directly from RGB-D images in about a second. We achieve this by training in an end-to-end fashion a coarse-to-fine model composed of a classification network that distinguishes grasp types according to a specific taxonomy and a refinement network that produces refined grasp poses and joint angles. We experimentally validate and benchmark our method against a standard grasp-sampling method on 790 grasps in simulation and 20 grasps on a real Franka Emika Panda. All experimental results using our method show consistent improvements both in terms of grasp quality metrics and grasp success rate. Remarkably, our approach is up to 20-30 times faster than the baseline, a significant improvement that opens the door to feedback-based grasp re-planning and task informative grasping. Code is available at https://irobotics.aalto.fi/multi-fingan/.

Domain adaptation is a common problem in robotics, with applications such as transferring policies from simulation to real world and lifelong learning. Performing such adaptation, however, requires informative data about the environment to be available during the adaptation. In this paper, we present domain curiosity—a method of training exploratory policies that are explicitly optimized to provide data that allows a model to learn about the unknown aspects of the environment. In contrast to most curiosity methods, our approach explicitly rewards learning, which makes it robust to environment noise without sacrificing its ability to learn. We evaluate the proposed method by comparing how much a model can learn about environment dynamics given data collected by the proposed approach, compared to standard curious and random policies. The evaluation is performed using a toy environment, two simulated robot setups, and on a real-world haptic exploration task. The results show that the proposed method allows data-efficient and accurate estimation of dynamics.

Enabling diverse users to program robots for different applications is critical for robots to be widely adopted. Most of the new collaborative robot manipulators come with intuitive programming interfaces that allow novice users to compose robot programs and tune their parameters. However, parameters like motion speeds or exerted forces cannot be easily demonstrated and often require manual tuning, resulting in a tedious trial-and-error process. To address this problem, we formulate tuning of one-dimensional parameters as an Active Learning problem where the learner iteratively refines its estimate of the feasible range of parameter values, by selecting informative queries. By executing the parametrized actions, the learner gathers the user's feedback, in the form of directional answers ("higher,'' "lower,'' or "fine''), and integrates it in the estimate. We propose an Active Learning approach based on Expected Divergence Maximization for this setting and compare it against two baselines with synthetic data. We further compare the approaches on a real-robot dataset obtained from programs written with a simple Domain-Specific Language for a robot arm and manually tuned by expert users (N=8) to perform four manipulation tasks. We evaluate the effectiveness and usability of our interactive tuning approach against manual tuning with a user study where novice users (N=8) tuned parameters of a human-robot hand-over program.

Deep reinforcement learning has proven to be successful for learning tasks in simulated environments, but applying same techniques for robots in real-world domain is more challenging, as they require hours of training. To address this, transfer learning can be used to train the policy first in a simulated environment and then transfer it to physical agent. As the simulation never matches reality perfectly, the physics, visuals and action spaces by necessity differ between these environments to some degree. In this work, we study how general video games can be directly used instead of fine-tuned simulations for the sim-to-real transfer. Especially, we study how the agent can learn the new action space autonomously, when the game actions do not match the robot actions. Our results show that the different action space can be learned by re-training only part of neural network and we obtain above 90% mean success rate in simulation and robot experiments.

In many robot control problems, factors such as stiffness and damping matrices and manipulability ellipsoids are naturally represented as symmetric positive definite (SPD) matrices, which capture the specific geometric characteristics of those factors. Typical learned skill models such as dynamic movement primitives (DMPs) can not, however, be directly employed with quantities expressed as SPD matrices as they are limited to data in Euclidean space. In this paper, we propose a novel and mathematically principled framework that uses Riemannian metrics to reformulate DMPs such that the resulting formulation can operate with SPD data in the SPD manifold. Evaluation of the approach demonstrates that beneficial properties of DMPs such as change of the goal during operation apply also to the proposed formulation.

Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance.

Current end-to-end grasp planning methods propose grasps in the order of seconds that attain high grasp success rates on a diverse set of objects, but often by constraining the workspace to top-grasps. In this work, we present a method that allows end-to-end top-grasp planning methods to generate full six-degree-of-freedom grasps using a single RGBD view as input. This is achieved by estimating the complete shape of the object to be grasped, then simulating different viewpoints of the object, passing the simulated viewpoints to an end-to-end grasp generation method, and finally executing the overall best grasp. The method was experimentally validated on a Franka Emika Panda by comparing 429 grasps generated by the state-of-the-art Fully Convolutional Grasp Quality CNN, both on simulated and real camera images. The results show statistically significant improvements in terms of grasp success rate when using simulated images over real camera images, especially when the real camera viewpoint is angled. Code and video are available at https://irobotics.aalto.fi/beyond-topgrasps-through-scene-completion/.

Safe yet stable grasping requires a robotic hand to apply sufficient force on the object to immobilize it while keeping it from getting damaged. Soft robotic hands have been proposed for safe grasping due to their passive compliance, but even such a hand can crush objects if the applied force is too high. Thus for safe grasping, regulating the grasping force is of uttermost importance even with soft hands. In this work, we present a force controlled soft hand and use it to achieve safe grasping. To this end, resistive force and bend sensors are integrated in a soft hand, and a data-driven calibration method is proposed to estimate contact interaction forces. Given the force readings, the pneumatic pressures are regulated using a proportional-integral controller to achieve desired force. The controller is experimentally evaluated and benchmarked by grasping easily deformable objects such as plastic and paper cups without neither dropping nor deforming them. Together, the results demonstrate that our force controlled soft hand can grasp deformable objects in a safe yet stable manner.

This paper investigates Active Robot Learning strategies that take into account the effort of the user in an interactive learning scenario. Most research claims that Active Learning's sample efficiency can reduce training time and therefore the effort of the human teacher. We argue that the performance driven query selection of standard Active Learning can make the job of the human teacher difficult, resulting in a decrease in training quality due to slowdowns or increased error rates. We investigate this issue by proposing a learning strategy that aims to minimize the user's workload by taking into account the flow of the questions. We compare this strategy against a standard Active Learning strategy based on uncertainty sampling and a third strategy being an hybrid of the two. After studying in simulation the validity and the behavior of these approaches, we conducted a user study where 26 subjects interacted with a NAO robot embodying the presented strategies. We reports results from both the robot's performance and the human teacher's perspectives, observing how the hybrid strategy represents a good compromise between learning performance and user's experienced workload. Based on the results, we provide recommendations on the development of Active Robot Learning strategies going beyond robot's performance.

We present a Learning from Demonstration method for teaching robots to perform search strategies imitated from humans in scenarios where alignment tasks fail due to position uncertainty. The method utilizes human demonstrations to learn both a state invariant dynamics model and an exploration distribution that captures the search area covered by the demonstrator. We present two alternative algorithms for computing a search trajectory from the exploration distribution, one based on sampling and another based on deterministic ergodic control. We augment the search trajectory with forces learnt through the dynamics model to enable searching both in force and position domains. An impedance controller with superposed forces is used for reproducing the learnt strategy. We experimentally evaluate the method on a KUKA LWR4+ performing a 2D peg-in-hole and a 3D electricity socket task. Results show that the proposed method can, with only few human demonstrations, learn to complete the search task.

In this paper we show how different choices regarding compliance affect a dual-arm assembly task. In addition, we present how the compliance parameters can be learned from a human demonstration. Compliant motions can be used in assembly tasks to mitigate pose errors originating from, for example, inaccurate grasping. We present analytical background and accompanying experimental results on how to choose the center of compliance to enhance the convergence region of an alignment task. Then we present the possible ways of choosing the compliant axes for accomplishing alignment in a scenario where orientation error is present. We show that an earlier presented Learning from Demonstration method can be used to learn motion and compliance parameters of an impedance controller for both manipulators. The learning requires a human demonstration with a single teleoperated manipulator only, easing the execution of demonstration and enabling usage of manipulators at difficult locations as well. Finally, we experimentally verify our claim that having both manipulators compliant in both rotation and translation can accomplish the alignment task with less total joint motions and in shorter time than moving one manipulator only. In addition, we show that the learning method produces the parameters that achieve the best results in our experiments.

Training end-to-end deep robot policies requires a lot of domain-, task-, and hardware-specific data, which is often costly to provide. In this work, we propose to tackle this issue by employing a deep neural network with a modular architecture, consisting of separate perception, policy, and trajectory parts. Each part of the system is trained fully on synthetic data or in simulation. The data is exchanged between parts of the system as low-dimensional latent representations of affordances and trajectories. The performance is then evaluated in a zero-shot transfer scenario using Franka Panda robot arm. Results demonstrate that a low-dimensional representation of scene affordances extracted from an RGB image is sufficient to successfully train manipulator policies. We also introduce a method for affordance dataset generation, which is easily generalizable to new tasks, objects and environments, and requires no manual pixel labeling.

Contextual skill models are learned to provide skills over a range of task parameters, often using regression across optimal task-specific policies. However, the sequential nature of the learning process is usually neglected. In this paper, we propose to use active incremental learning by selecting a task which maximizes performance improvement over entire task set. The proposed framework exploits knowledge of individual tasks accumulated in a database and shares it among the tasks using a contextual skill model. The framework is agnostic to the type of policy representation, skill model, and policy search. We evaluated the skill improvement rate in two tasks, ball-in-a-cup and basketball. In both, active selection of tasks lead to a consistent improvement in skill performance over a baseline.

We present a method for planning robust grasps over uncertain shape completed objects. For shape completion, a deep neural network is trained to take a partial view of the object as input and outputs the completed shape as a voxel grid. The key part of the network is dropout layers which are enabled not only during training but also at run-time to generate a set of shape samples representing the shape uncertainty through Monte Carlo sampling. Given the set of shape completed objects, we generate grasp candidates on the mean object shape but evaluate them based on their joint performance in terms of analytical grasp metrics on all the shape candidates. We experimentally validate and benchmark our method against another state-of-the-art method with a Barrett hand on 90000 grasps in simulation and 200 grasps on a real Franka Emika Panda. All experimental results show statistically significant improvements both in terms of grasp quality metrics and grasp success rate, demonstrating that planning shape-uncertainty-aware grasps brings significant advantages over solely planning on a single shape estimate, especially when dealing with complex or unknown objects.

Accurate manipulation of a deformable body such as a piece of fabric is difficult because of its many degrees of freedom and unobservable properties affecting its dynamics. To alleviate these challenges, we propose the application of feedback-based control to robotic fabric strip folding. The feedback is computed from the low dimensional state extracted from a camera image. We trained the controller using reinforcement learning in simulation which was calibrated to cover the real fabric strip behaviors. The proposed feedback-based folding was experimentally compared to two state-of-the-art folding methods and our method outperformed both of them in terms of accuracy.

We present the participatory design process of a robotic tutor of assistive sign language for children with autism spectrum disorder (ASD). Robots have been used in autism therapy, and to teach sign language to neurotypical children. The application of teaching assistive sign language - the most common form of assistive and augmentative communication used by people with ASD - is novel. The robot's function is to prompt children to imitate the assistive signs that it performs. The robot was therefore co-designed to appeal to children with ASD, taking into account the characteristics of ASD during the design process: impaired language and communication, impaired social behavior, and narrow flexibility in daily activities. To accommodate these characteristics, a multidisciplinary team defined design guidelines specific to robots for children with ASD, which were followed in the participatory design process. With a pilot study where the robot prompted children to imitate nine assistive signs, we found support for the effectiveness of the design. The children successfully imitated the robot and kept their focus on it, as measured by their eye gaze. Children and their companions reported positive experiences with the robot, and companions evaluated it as potentially useful, suggesting that robotic devices could be used to teach assistive sign language to children with ASD.

Transparency of robot behaviors increases efficiency and quality of interactions with humans. To increase transparency of robot policies, we propose a method for generating robust and focused explanations that express why a robot chose a particular action. The proposed method examines the policy based on the state space in which an action was chosen and describes it in natural language. The method can generate focused explanations by leaving out irrelevant state dimensions, and avoid explanations that are sensitive to small perturbations or have ambiguous natural language concepts. Furthermore, the method is agnostic to the policy representation and only requires the policy to be evaluated at different samples of the state space. We conducted a user study with 18 participants to investigate the usability of the proposed method compared to a comprehensive method that generates explanations using all dimensions. We observed how focused explanations helped the subjects more reliably detect the irrelevant dimensions of the explained system and how preferences regarding explanation styles and their expected characteristics greatly differ among the participants.

In a multirobot system, using shared symbols for objects in the environment is a prerequisite for collaboration. Sharing symbols requires that each agent has anchored a symbol with an internal, sensor level representation, as well as that these symbols match between the agents. The problem can be solved easily when the internal representations can be communicated between the agents. However, with heterogeneous embodiments the available sensors are likely to differ, making it impossible to share the internal representations directly. We propose the use of pointing gestures to align symbols between a heterogeneous group of robots. We describe a planning framework that minimizes the required effort for anchoring representations across robots. The framework allows planning for both the gesturing and observing agents in a decentralized fashion. It considers both implicit sources of failure, such as ambiguous pointing, as well as costs required by actions. Simulation experiments demonstrate that the resulting planning problem has a complex solution structure with multiple local minima. Demonstration with a heterogeneous two-robot system shows the practical viability of this approach.

With the goal of having robots learn new skills after deployment, we propose an active learning framework for modelling user preferences about task execution. The proposed approach interactively gathers information by asking questions expressed in natural language. We study the validity and the learning performance of the proposed approach and two of its variants compared to a passive learning strategy. We further investigate the human-robot-interaction nature of the framework conducting a usability study with 18 subjects. The results show that active strategies are applicable for learning preferences in temporal tasks from non-expert users. Furthermore, the results provide insights in the interaction design of active learning robots.

To cope with varying conditions, motor primitives (MPs) must support generalization over task parameters to avoid learning separate primitives for each situation. In this regard, deterministic and probabilistic models have been proposed for generalizing MPs to new task parameters, thus providing limited generalization. Although generalization of MPs using probabilistic models has been studied, it is not clear how such generalizable models can be learned efficiently. Reinforcement learning can be more efficient when the exploration process is tuned with data uncertainty, thus reducing unnecessary exploration in a data-efficient way. We propose an empirical Bayes method to predict uncertainty and utilize it for guiding the exploration process of an incremental learning framework. The online incremental learning framework uses a single human demonstration for constructing a database of MPs. The main ingredients of the proposed framework are a global parametric model (GPDMP) for generalizing MPs for new situations, a model-free policy search agent for optimizing the failed predicted MPs, model selection for controlling the complexity of GPDMp, and empirical Bayes for extracting the uncertainty of MPs prediction. Experiments with a ball-in-a-cup task demonstrate that the global GPDMP model generalizes significantly better than linear models and Locally Weighted Regression especially in terms of extrapolation capability. Furthermore, the model selection has successfully identified the required complexity of GPDMP even with few training samples while satisfying the Occam Razor's prinicple. Above all, the uncertainty predicted by the proposed empirical Bayes approach successfully guided the exploration process of the model-free policy search. The experiments indicated statistically significant improvement of learning speed over covariance matrix adaptation (CMA) with a significance of p=0.002.

Compliant motions allow alignment of workpieces using naturally occurring interaction forces. However, free-floating objects do not have a fixed base to absorb the reaction forces caused by the interactions. Consequently, if the interaction forces are too high, objects can gain momentum and move away after contact. This paper proposes an approach based on direct force control for compliant manipulation of free-floating objects. The objective of the controller is to minimize the interaction forces while maintaining the contact. The proposed approach achieves this by maintaining small constant force along the motion direction and an apparent reduction of manipulator inertia along remaining Degrees of Freedom (DOF). Simulation results emphasize the importance of relative inertia of the robotic manipulator with respect to the free-floating object. The experiments were performed with KUKA LWR4+ manipulator arm and a two-dimensional micro-gravity emulator (object floating on an air bed), which was developed in-house. It was verified that the proposed control law is capable of controlling the interaction forces and aligning the tools without pushing the object away. We conclude that direct force control works better with a free-floating object than implicit force control algorithms, such as impedance control.

In near future, robots are envisioned to work alongside humans in unstructured professional and domestic environments. In such setups, collaborative manipulation is a fundamental skill that allows manipulation of heavy loads by load sharing between agents. Grasp planning plays a pivotal role for load sharing but it has not received attention in the literature. This work proposes a grasp analysis approach for collaborative manipulation that allows load sharing by minimizing exerted grasp wrenches in a task specific way. The manipulation task is defined as expected external wrenches acting on the target object. The analysis approach is demonstrated in a two-agent decentralized set-up with unknown objects. After the first agent has grasped the target, the second agent observes the first agent's grasp location and plans its own grasp according to optimal load sharing. The method was verified in a human robot collaborative lifting task. Experiments with multiple objects show that the proposed method results in optimal load sharing despite limited information and partial observability.

This paper proposes an approach for segmenting a task consisting of compliant motions into phases, learning a primitive for each segmented phase of the task, and reproducing the task by sequencing primitives online based on the learned model. As compliant motions can “probe” the environment, using the interaction between the robot and the environment to detect phase transitions can make the transitions less prone to positional errors. This intuition leads us to model a task with a non-homogeneous Hidden Markov Model (HMM), wherein hidden phase transition probabilities depend on the interaction with the environment (wrench measured by an F/T sensor). Expectation-maximization algorithm is employed in estimating the parameters of the HMM model. During reproduction, the phase changes of a task are detected online using the forward algorithm, with the parameters learned from demonstrations. Cartesian impedance controller parameters are learned from the demonstrations to reproduce each phase of the task. The proposed approach is studied with a KUKA LWR4+ arm in two setups. Experiments show that the method can successfully segment and reproduce a task consisting of compliant motions with one or more demonstrations, even when demonstrations do not have the same starting position and external forces occur from different directions. Finally, we demonstrate that the method can also handle rotational motions.

Many mobile robots rely on 2D laser scanners for localization, mapping, and navigation. However, those sensors are unable to correctly provide distance to obstacles such as glass panels and tables whose actual occupancy is invisible at the height the sensor is measuring. In this work, instead of estimating the distance to obstacles from richer sensor readings such as 3D lasers or RGBD sensors, we present a method to estimate the distance directly from raw 2D laser data. To learn a mapping from raw 2D laser distances to obstacle distances we frame the problem as a learning task and train a neural network formed as an autoencoder. A novel configuration of network hyperparameters is proposed for the task at hand and is quantitatively validated on a test set. Finally, we qualitatively demonstrate in real time on a Care-O-bot 4 that the trained network can successfully infer obstacle distances from partial 2D laser readings.

This paper presents, for the first time, a method for learning in-contact tasks from a teleoperated demonstration with a hydraulic manipulator. Due to the use of extremely powerful hydraulic manipulator, a force-reflected bilateral teleoperation is the most reasonable method of giving a human demonstration. An advanced subsystem-dynamic-based control design framework, virtual decomposition control (VDC), is used to design a stability-guaranteed controller for the teleoperation system, while taking into account the full nonlinear dynamics of the master and slave manipulators. The use of fragile force/torque sensor at the tip of the hydraulic slave manipulator is avoided by estimating the contact forces from the manipulator actuators' chamber pressures. In the proposed learning method, it is observed that a surface-sliding tool has a friction-dependent range of directions (between the actual direction of motion and the contact force) from which the manipulator can apply force to produce the sliding motion. By this intuition, an intersection of these ranges can be taken over a motion to robustly find a desired direction for the motion from one or more demonstrations. The compliant axes required to reproduce the motion can be found by assuming that all motions outside the desired direction is caused by the environment, signalling the need for compliance. Finally, the learning method is incorporated to a novel VDC-based impedance control method to learn compliant behaviour from teleoperated human demonstrations. Experiments with 2-DOF hydraulic manipulator with a 475kg payload demonstrate the suitability and effectiveness of the proposed method to perform learning from demonstration (LfD) with heavy-duty hydraulic manipulators.

Robotic systems developed for support can provide assistance in various ways. However, regardless of the service provided, the quality of user interaction is key to adoption by the general public. Simple communication difficulties, such as terminological differences, can make or break the acceptance of robots. In this work we take into account these difficulties in communication between a human and a robot. We propose a system that allows to handle unknown concepts through symbol manipulation based on natural language interactions. In addition, ontologies are used as a convenient way to store the knowledge and reason about it. To demonstrate the use of our system, two scenarios are described and tested with a Care-O-Bot 4. The experiments show that confusions and difficulties in communication can effectively be resolved through symbol manipulation.

Trajectory optimization is a fundamental problem in robotics. While optimization of continuous control trajectories is well developed, many applications require both discrete and continuous, i.e. hybrid controls. Finding an optimal sequence of hybrid controls is challenging due to the exponential explosion of discrete control combinations. Our method, based on Differential Dynamic Programming (DDP), circumvents this problem by incorporating discrete actions inside DDP: we first optimize continuous mixtures of discrete actions, and, subsequently force the mixtures into fully discrete actions. Moreover, we show how our approach can be extended to partially observable Markov decision processes (POMDPs) for trajectory planning under uncertainty. We validate the approach in a car driving problem where the robot has to switch discrete gears and in a box pushing application where the robot can switch the side of the box to push. The pose and the friction parameters of the pushed box are initially unknown and only indirectly observable.

This paper investigates manipulation of multiple unknown objects in a crowded environment. Because of incomplete knowledge due to unknown objects and occlusions in visual observations, object observations are imperfect and action success is uncertain, making planning challenging. We model the problem as a partially observable Markov decision process (POMDP), which allows a general reward based optimization objective and takes uncertainty in temporal evolution and partial observations into account. In addition to occlusion dependent observation and action success probabilities, our POMDP model also automatically adapts object specific action success probabilities. To cope with the changing system dynamics and performance constraints, we present a new online POMDP method based on particle filtering that produces compact policies. The approach is validated both in simulation and in physical experiments in a scenario of moving dirty dishes into a dishwasher. The results indicate that: 1) a greedy heuristic manipulation approach is not sufficient, multi-object manipulation requires multi-step POMDP planning, and 2) on-line planning is beneficial since it allows the adaptation of the system dynamics model based on actual experience.

Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sample-efficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method uses Variational Inference to estimate policy parameters, while at the same time uncovering a low-dimensional latent space of controls. Prior knowledge about the task and the structure of the learning agent can be provided by specifying groups of potentially correlated parameters. This information is then used to impose sparsity constraints on the mapping between the high-dimensional space of controls and a lower-dimensional latent space. In experiments with a simulated bi-manual manipulator, the new approach effectively identifies synergies between joints, performs efficient low-dimensional policy search, and outperforms state-of-the-art policy search methods.

Kinesthetic teaching is an established method of teaching robots new skills without requiring robotics or programming knowledge. However, the inertia and uncoordinated motions of individual joints decrease the intuitiveness and naturalness of interaction and impair the quality of the learned skill. This paper proposes a method to ease kinesthetic teaching by combining the idea of incremental learning through warping several demonstrations into a common frame with virtual tool dynamics to assist the user during teaching. In fact, during a sequence of demonstrations the stiffness of the robot under Cartesian impedance control is gradually increased, to provide stronger assistance to the user based on the demonstrations accumulated up to that moment. Therefore, the operator has the opportunity to progressively refine the task's model while the robot more docilely follows the learned action. Robot experiments and a user study performed on 25 novice users show that the proposed approach improves both usability as well as resulting skill quality.

Imitation learning of tasks in multi-component robotic systems requires capturing concurrency and synchronization requirements in addition to task structure. Learning time-critical tasks depends furthermore on the ability to model temporal elements in demonstrations. This paper proposes a modeling framework based on Petri nets capable of modeling these aspects in a programming by demonstration context. In the proposed approach, models of tasks are constructed from segmented demonstrations as task Petri nets, which can be executed as discrete controllers for reproduction. We present algorithms that automatically construct models from demonstrations, showing how elements of time-critical tasks can be mapped into task Petri net elements. The approach is validated by an experiment in which a robot plays a musical passage on a keyboard.

Learning to perform tasks like pulling a door handle or pushing a button, inherently easy for a human, can be surprisingly difficult for a robot. A crucial problem in these kinds of in-contact tasks is the context specificity of pose and force requirements. In this paper, a robot learns in-contact tasks from human kinesthetic demonstrations. To address the need to balance between the position and force constraints, we propose a model based on the hidden semi-Markov model (HSMM) and Cartesian impedance control. The model captures uncertainty over time and space and allows the robot to smoothly satisfy a task's position and force constraints by online modulation of impedance controller stiffness according to the HSMM state belief. In experiments, a KUKA LWR 4+ robotic arm equipped with a force/torque sensor at the wrist successfully learns from human demonstrations how to pull a door handle and push a button.

We present SaDDP, a sampled version of the widely used differential dynamic programming (DDP) control algorithm. We contribute through establishing a novel connection between two major branches of robotics control research, that is, gradient-based methods such as DDP, and Monte Carlo methods such as path integral control (PI) that utilize random simulated trajectory rollouts. One of our key observations is that the Taylor-expansion, central to DDP, can be reformulated in terms of second-order statistics computed of the sampled trajectories. SaDDP makes few assumptions about the controlled system and works with black-box dynamics simulations with non-smooth contacts. Our simulation results show that the method outperforms PI and CMA-ES in both a simple linear-quadratic problem, and a multilink arm reaching task with obstacles.

Grasping systems that build upon meticulously planned hand postures rely on precise knowledge of object geometry, mass and frictional properties — assumptions which are often violated in practice. In this work, we propose an alternative solution to the problem of grasp acquisition in simple autonomous pick and place scenarios, by utilizing the concept of grasp envelopes: sets of constraints on gripper postures. We propose a fast method for extracting grasp envelopes for objects that fit within a known shape category, placed in an unknown environment. Our approach is based on grasp envelope primitives, which encode knowledge of human grasping strategies. We use environment models, reconstructed from noisy sensor observations, to refine the grasp envelope primitives and extract bounded envelopes of collision-free gripper postures. Also, we evaluate the envelope extraction procedure both in a stand alone fashion, as well as an integrated component of an autonomous picking system.

Automating assembly processes outside controlled factory environments is still rare, mostly because of the inherent position uncertainties. The use of compliant motions allows robustness against the uncertainty, but automatic planning of compliant motion sequences is not computationally feasible. In this paper, we show how compliant assembly motions can be learned from human demonstrations. A human teacher will kinesthetically demonstrate compliant motions where the physical shapes of assembled parts guide the motion. From these demonstrations, the proposed method identifies desired direction of movement, the number of compliant axes and their directions. We use this information to construct an impedance controller which can reproduce the assembly motion despite uncertainty in the starting position. The method is studied with a KUKA LWR4+ arm in two test setups with different number of physically constrained degrees of freedom. The experimental study shows that the method is able to correctly identify the motion parameters and allows the robot to successfully perform the demonstrated assembly motion from various unseen starting positions.

We present a (generalizable) method aimed to simultaneously transfer positional and force requirements encoded in a physical human skill (wood planing) from a human instructor to a robotic arm through kinesthetic teaching. We achieve our goal through a novel use of a common sensory configuration, constituted by a force/torque sensor mounted between the tool and the flange of a robotic arm. The robotic arm is endowed with integrated torque sensors at each joint. The mathematical model used to capture the general dynamic of the interaction between the human user and the wood surface is based on Dynamic Movement Primitives. During reenactment of the task, the system can imitate and generalize the demonstrated spatial requirements, as well as their associated force profiles. Therefore, the robotic arm acquires the capacity to reproduce the dynamic profile for in-contact tasks requiring an articulated coordination in the distribution of forces. For example, the capacity to effectively operate the plane on a wood plank over multiple strokes, according to the demonstration of the human instructor.

Grasp planning in multi-robot systems is usually studied in a centralized setting with all robots sharing common knowledge about the overall system. Relaxing this assumption would allow multiple mobile manipulators to cooperate even without strict and precise coordination. Moreover, most typical tasks for cooperative settings, such as transporting heavy objects, require certain forces/torques to be exerted along/around particular directions, for instance, compensating for the weight of the transported object. In this paper, we propose task specific multi-robot grasp planning strategies that allow decentralized planning. Each agent plans its own actions without precise information about the other's plans. The approach is based on analysing a task specific grasp quality metric in a probabilistic context, compensating thus for the incomplete knowledge. Results from simulation experiments demonstrate that task independent planning is clearly inferior when task characteristics are known and thus task specific quality measures should be used. Furthermore, the proposed decentralized planning approaches clearly outperform the baseline and show close to globally optimal performance.

Making decisions based on visual input is challenging because determining how the scene should be split into individual objects is often very difficult. While previous work mainly considers decision making and visual processing as two separate tasks, we argue that the inherent uncertainty in object segmentation requires an integrated approach that chooses the best decision over all possible segmentations. Our approach over-segments the visual input and combines the segments into possible objects to get a probability distribution over object compositions, represented as particles. We introduce a Markov chain Monte Carlo procedure that aims to produce exact, independent samples. In experiments, where a 6-DOF robot arm moves object hypotheses captured by an RGB-D visual sensor, our approach of probability distribution based decision making outperforms an approach which utilises the traditional most likely object composition.

Mobile manipulation of large objects can benefit greatly from the use of multiple cooperating robots. Multi-robot coordination of decentralized systems is, however, challenging due to the nature of such systems. For this reason, many planning problems are yet unexplored. This paper proposes two decentralized approaches for cooperative grasp planning. In our setting, agents do not have information about the embodiments of other agents and each agent thus needs to plan its own actions. The proposed approaches are based on analysing the traditional grasp quality metrics in a probabilistic context which compensates for the incomplete information. Results from simulation experiments demonstrate that the probabilistic approaches clearly improve performance over a baseline approach where the grasps are planned independently. Moreover, their performance is close to that of the optimal centralized approach.

This paper addresses the idea of establishing symbolic communication between mobile robots through gesturing. Humans communicate using body language and gestures in addition to other linguistic modalities like prosody and text or dialog structure. This research aims to develop a pointing gesture detection system for robot to robot communication scenarios to grant robots an ability to convey object identity information without global localization of the agents. The detection is based on RGB-D and a NAO humanoid robot is used as the pointing agent in the experiments. The presented algorithms are based on PCL library. The results indicate that real-time detection of pointing gesture can be performed with little information about the embodiment of the pointing agent and that an observing agent can use the gesture detection to perform actions on the pointed targets.

Manipulating unknown objects in a cluttered environment is difficult because object composition is uncertain. Because of this uncertainty, earlier work has concentrated on finding the “best” object composition and based on this composition decided on manipulation actions. Contrary to earlier work, we 1) utilize different possible object compositions in decision making, 2) take advantage of object composition information provided by robot actions, 3) take into account the effect of different competing object hypothesis on the actual task to be performed. We cast the manipulation planning problem as a partially observable Markov decision process (POMDP) which plans over possible hypotheses of object compositions. The POMDP model chooses the action that maximizes the long-term expected task specific utility, and while doing so, considers the value of informative actions and the effect of different object hypotheses on the completion of the task. In experiments with a physical robot arm and an RGB-D sensor, our approach outperforms an approach that only considers the most likely object composition.

This paper presents a physical human-robot interaction (pHRI) interface, which enables the user to control a walking humanoid robot through physical contact. A human operator guides the robot in parent-child-like behavior by exerting force onto the hand of the robot. In contrast to a conventional approach of pHRI in which force/torque measurements are applied, the proposed solution is based on sensorless force control. Furthermore, we demonstrate an extension to the generic interface by implementing a number of gait control algorithms. This paper also evaluates user-acceptable robot responses while guiding a humanoid robot walking. User opinion on the presented control solution is evaluated through usability testing conducted among prospective users. The obtained results indicate that the developed interfaces present an effective solution to the problem of sensorless guidance of a walking robot.

We present a Model-Predictive Control (MPC) system for online synthesis of interactive and physically valid character motion. Our system enables a complex (36-DOF) 3D human character model to balance in a given pose, dodge projectiles, and improvise a get up strategy if forced to lose balance, all in a dynamic and unpredictable environment. Such contact-rich, predictive and reactive motions have previously only been generated offline or using a handcrafted state machine or a dataset of reference motions, which our system does not require. For each animation frame, our system generates trajectories of character control parameters for the near future --- a few seconds --- using Sequential Monte Carlo sampling. Our main technical contribution is a multimodal, tree-based sampler that simultaneously explores multiple different near-term control strategies represented as parameter splines. The strategies represented by each sample are evaluated in parallel using a causal physics engine. The best strategy, as determined by an objective function measuring goal achievement, fluidity of motion, etc., is used as the control signal for the current frame, but maintaining multiple hypotheses is crucial for adapting to dynamically changing environments.

Unmanned autonomous systems offer safety benefits for potentially hazardous environments. The evaluation of the performance of such systems is challenging because hazardous environments, i.e. due to flood or fire, are constantly changing. In this paper we introduce a system for simulation-based risk assessment of a fleet of autonomous machines performing a rescue mission. The simulation is based on combining the dynamic propagation of the hazard and the real-time simulation of the fleet. Our use case analyses an underground rescue mission under a pipeline breakage. The propagation of the flood is simulated together with the actions of robots. We evaluate the system by simulating several scenarios and measuring the mission success for each. As a result we show that a simulation based risk assessment is a feasible option for evaluating the performance of complex distributed systems.

In this work, we propose to reconstruct a complete 3-D model of an unknown object by fusion of visual and tactile information while the object is grasped. Assuming the object is symmetric, a first hypothesis of its complete 3-D shape is generated from a single view. This initial model is used to plan a grasp on the object which is then executed with a robotic manipulator equipped with tactile sensors. Given the detected contacts between the fingers and the object, the full object model including the symmetry parameters can be refined. This refined model will then allow the planning of more complex manipulation tasks. The main contribution of this work is an optimal estimation approach for the fusion of visual and tactile data applying the constraint of object symmetry. The fusion is formulated as a state estimation problem and solved with an iterative extended Kalman filter. The approach is validated experimentally using both artificial and real data from two different robotic platforms.

In this paper, we present a novel probabilistic framework for grasping. In the framework, grasp and object attributes, on-line sensor information and the stability of a grasp are all considered through probabilistic models. We describe how sensor-based grasp planning can be formulated in a probabilistic framework and how information about object attributes can be updated simultaneously using on-line sensor information gained during grasping. The framework is demonstrated by building the necessary probabilistic models using Gaussian process regression, and using the models with an MCMC approach to estimate a target object's pose and grasp stability during grasp attempts. The framework is also demonstrated on a real robotic platform.

Learning from demonstration is an invaluable skill for a robot acting in a human populated natural environment, allowing the teaching of new skills without tedious and complex manual programming. Physical human-robot interaction, where the human is in a physical contact with the robot, is a promising approach for teaching especially manipulation skills. This paper studies the human side of physical human-robot interaction, in the context of a human physically guiding a robot through the desired set of motions. The paper addresses the question, which kind of response of the robot is preferable for the human user. In addition, different approaches for the guidance are described and relevant technical challenges are discussed. The main finding of the user study is that there is a need for a trade-off between the conflicting goals of naturalness of motion and positioning accuracy.

An important ability of a robot that interacts with the environment and manipulates objects is to deal with the uncertainty in sensory data. Sensory information is necessary to, for example, perform online assessment of grasp stability. We present methods to assess grasp stability based on haptic data and machine-learning methods, including AdaBoost, support vector machines (SVMs), and hidden Markov models (HMMs). In particular, we study the effect of different sensory streams to grasp stability. This includes object information such as shape; grasp information such as approach vector; tactile measurements from fingertips; and joint configuration of the hand. Sensory knowledge affects the success of the grasping process both in the planning stage (before a grasp is executed) and during the execution of the grasp (closed-loop online control). In this paper, we study both of these aspects. We propose a probabilistic learning framework to assess grasp stability and demonstrate that knowledge about grasp stability can be inferred using information from tactile sensors. Experiments on both simulated and real data are shown. The results indicate that the idea to exploit the learning approach is applicable in realistic scenarios, which opens a number of interesting venues for the future research.

The adoption of robots for service tasks in natural environments calls for the use of sensors to allow manipulation of objects under imperfect environment knowledge and the use of knowledge transfer from humans. This paper addresses these challenges by proposing a new abstraction architecture for embodiment independent sensor-based control of manipulation. The aim is to address three specific challenges: hardware independent control of manipulation, use of sensors to alleviate problems of complexity and uncertainty of the environment, and ease of transferring knowledge over different embodiments through a hierarchical abstract representation of manipulation skills. The proposed abstraction architecture is demonstrated for hardware independence and failure detection on two different manipulator platforms.

This paper studies the learning of task constraints that allow grasp generation in a goal-directed manner. We show how an object representation and a grasp generated on it can be integrated with the task requirements. The scientific problems tackled are (i) identification and modeling of such task constraints, and (ii) integration between a semantically expressed goal of a task and quantitative constraint functions defined in the continuous object-action domains. We first define constraint functions given a set of object and action attributes, and then model the relationships between object, action, constraint features and the task using Bayesian networks. The probabilistic framework deals with uncertainty, combines a-priori knowledge with observed data, and allows inference on target attributes given only partial observations. We present a system designed to structure data generation and constraint learning processes that is applicable to new tasks, embodiments and sensory data. The application of the task constraint model is demonstrated in a goal-directed imitation experiment.

In this paper, the problem of learning grasp stability in robotic object grasping based on tactile measurements is studied. Although grasp stability modeling and estimation has been studied for a long time, there are few robots today able of demonstrating extensive grasping skills. The main contribution of the work presented here is an investigation of probabilistic modeling for inferring grasp stability based on learning from examples. The main objective is classification of a grasp as stable or unstable before applying further actions on it, e.g. lifting. The problem cannot be solved by visual sensing which is typically used to execute an initial robot hand positioning with respect to the object. The output of the classification system can trigger a regrasping step if an unstable grasp is identified. An off-line learning process is implemented and used for reasoning about grasp stability for a three-fingered robotic hand using Hidden Markov models. To evaluate the proposed method, experiments are performed both in simulation and on a real robot system.

It has recently been demonstrated that the effect of visual measurement errors on the open-loop control in visual servoing can be estimated using linear propagation of errors. The uncertainty estimation offers a tool to build and analyze hybrid control systems such as switching or partitioning control. In earlier works, position-based and 2.5D servoing have been analysed. This work extends the approach to the analysis of closed loop uncertainty, showing how the path uncertainty can be approximated. The uncertainty is analyzed in Cartesian reference to make the approach general over different hardware. In addition, we show how image-based visual servoing can be analysed using the same approach.

Photometric stereo can be used to obtain a fast and noncontact surface reconstruction of Lambertian surfaces. Despite several published works concerning the uncertainties and optimal light configurations of photometric stereo, no solutions for optimal surface reconstruction from noisy real images have been proposed. In this paper, optimal surface reconstruction methods for approximate planar textured surfaces using photometric stereo are derived, given that the statistics of imaging errors are measurable. Simulated and real surfaces are experimentally studied, and the results validate that the proposed approaches improve the surface reconstruction especially for the high-frequency height variations.

We show how matching and reconstruction of contour points can be performed using dynamic time warping (DTW) for the purpose of 3D hand contour tracking. We evaluate the performance of the proposed algorithm in object manipulation activities and perform comparison with the iterative closest point (ICP) method.

The use of hierarchical maps has recently been proposed as an approach to make real-time visual SLAM computationally tractable in large environments. Compared to earlier work in SLAM with other sensors, local maps generated from monocular vision have one significant difference, namely, their scale is varying over different local maps. Thus, the relationship between two maps is a similarity transformation, and not an Euclidean transformation. This paper presents a novel representation of similarity transformations using quaternions. The representation is complete, is not overparameterized, does not have singularities of consequence, and its maximum likelihood estimation can be performed in real-time. The paper describes the parameterized transformation, as well as how the parameters can be optimized in real-time in a monocular visual SLAM system. Experiments presented show the validity of the approach.

In this paper, we propose and study surface reconstruction techniques for surfaces with high frequency height variation, which are common for example, in paper and textile manufacturing. Traditionally, photometric stereo methods have been developed and evaluated on objects with additive Gaussian noise. The minimization based methods may perform well on large objects, but they smooth the inherent high frequency variation of machined surfaces in the reconstruction. We extend a Fourier integration method with Wiener filter to reconstruct surfaces from two gradient fields. The experimental results validate that the proposed method performs well on surfaces with high frequency height variation.

We investigate modeling and recognition of arm manipulation actions of different levels of complexity. To model the process, we are using a combination of discriminative support vector machines and generative hidden Markov models. The experimental evaluation, performed with 10 people, investigates both definition and structure of primitive motions as well as the validity of the modeling approach taken.

Sensor-based robotic manipulation is becoming more and more popular as it promises increases in productivity, flexibility, and robustness of manipulation. Combining visual and force sensing is currently one of the most promising approaches for sensor-based manipulation, as vision and force are two complementary sensing modalities. One approach for multi-sensor use is the traded control where the robot is at each time controlled using one sensing modality, and the controllers are switched based on sensory input. One of the major problems with such systems is the transition between visual and force controllers. In this paper, we present a smooth transition method from motion to force control. The velocity of the end-effector is controlled by estimating the distance to the target by vision and determining an optimal velocity profile giving rapid approach and minimal force overshoot. Experiments show that the proposed control scheme is superior to earlier approaches

Model based object tracking has earned significant importance in areas such as augmented reality, surveillance, visual servoing, robotic object manipulation and grasping. Although an active research area, there are still few systems that perform robustly in realistic settings. The key problems to robust and precise object tracking are outliers caused by occlusion, self-occlusion, cluttered background, and reflections. Two most common solutions to the above problems have been the use of robust estimators and the integration of visual cues. The tracking system considered in this paper achieves robustness by integrating model-based and model-free cues. As model-based cues, we consider a CAD model of the object known a priori and as model-free cues, automatically generated corner features are used. The main idea is to account for relative object motion between consecutive frames using integration of the two cues. The particular contribution of this work is the integration framework where not only polyhedral objects are considered. In particular, we deal with spherical, cylindrical and conical objects for which the complete pose cannot be estimate using only CAD like models. Using the integration with the model-free features, we show how a full pose estimate can be obtained. Experimental evaluation demonstrates robust system performance in realistic settings with highly textured objects

For almost three decades the use of features based on Gabor filters has been promoted for their useful properties in image processing. The most important properties are related to invariance to illumination, rotation, scale, and translation. These properties are based on the fact that they are all parameters of Gabor filters themselves. This is especially useful in feature extraction, where Gabor filters have succeeded in many applications, from texture analysis to iris and face recognition. This study provides an overview of Gabor filters in image processing, a short literature survey of the most significant results, and establishes invariance properties and restrictions to the use of Gabor filters in feature extraction. Results are demonstrated by application examples.

Vision is one of the most powerful sensory modalities in robotics, allowing operation in dynamic envi ronments. One of our long-term research interests is mobile manipulation, where precise location of the target object is commonly required during task execution. Recently, a number of approaches have been proposed for real-time 3D tracking and most of them utilize an edge (wireframe) model of the target. However, the use of an edge model has significant problems in complex scenes due to occlusions and multiple responses, especially in terms of initialization. In this paper, we propose a new tracking method based on integration of model-based cues with automatically generated model-free cues, in order to improve tracking accuracy and to avoid weaknesses of edge based tracking. The integration is performed in a Kalman filter framework that operates in real-time. Experimental evaluation shows that the inclusion of model-free cues offers superior performance.

In recent years, a number of hybrid visual servoing control algorithms have been proposed and evaluated. For some time now, it has been clear that classical control approaches-image and position based-have some inherent problems. Hybrid approaches try to combine them to overcome these problems. However, most of the proposed approaches concentrate on the design of the control law, neglecting the issue of errors resulting from the sensory system. This paper addresses the issue of measurement errors in visual servoing. The particular contribution is the analysis of the propagation of image error through pose estimation and visual servoing control law. We have chosen to investigate the properties of the vision system and their effect to the performance of the control system. Two approaches are evaluated: i) position, and ii) 2 1/2 D visual servoing. We believe that our evaluation offers a tool to build and analyze hybrid control systems based on, for example, switching or partitioning.

In recent years, a number of visual servo control algorithms have been proposed. Most approaches try to solve the inherent problems of image-based and position based servoing by partitioning the control between image and Cartesian spaces. However, partitioning of the control often causes the Cartesian path to become more complex, which might result in operation close to the joint limits. A solution to avoid the joint limits is to use a shortest-path approach, which avoids the limits in most cases. In this paper, two new shortest-path approaches to visual servoing are presented. First, a position-based approach is proposed that guarantees both shortest Cartesian trajectory and object visibility. Then, a variant is presented, which avoids the use of a 3D model of the target object by using homography based partial pose estimation.

Automated visual inspection has become important for modern industry mainly because production rates and the level of automation have continually increased. The paper presents a system for automated visual inspection of large two-dimensional parts. The system is capable of inspecting sheet metal parts using CAD data. The inspection is performed based on a CAD model. There are existing systems that perform this function but they are not particularly well suitable for in-place inspection. In the proposed system, the high precision inspection of individual features is performed by firstly estimating the global position of a part. Next, each local feature is measured using subpixel techniques. Finally, the measurements are compared with a CAD model. Experiments are presented to evaluate the precision of system components. According to the results, the calibration procedure seems to be the factor that has the greatest effect on the final precision. Last, a comparison to similar systems is made and some suggestions are given how the precision could be further improved.