Haptic interaction plays a crucial role in enhancing realism and immersion in virtual environments, particularly in applications such as robotic rehabilitation. In these environments, therapy based on serious games that combine VR and haptic feed promise to offer engaging, affordable and reliable treatments. Most 6-DoF haptic methods have focused on precision in the kinematic fidelity of the haptic feedback. However, rehabilitation games for upper limb focus more on the forces exerted by the patient than the kinematic precision of the contacts with the virtual environment, since these exerted forces will be what powers the neuromuscular recovery of lost movement. We believe that a haptic control method with a focus on providing stable compliant and safe collisions is required to further the research into VR rehabilitation gamification. In addition, these solutions often rely on specifically tailored simulators running at very high frequencies. In contrast, commercial game engine that bring the most capabilities for gamification run at very low frequencies in comparison (≃50 Hz). In this work, we present a novel approach that integrates god object and penalty-based methods to achieve a balance between stability and computational efficiency, to enable 6-DoF haptic force generation on a collaborative robot. This is achieved through a relaxed quasi-god object simulation in the game engine and geodesic constraints in the robot’s haptic loop.

Robotic-assisted upper limb rehabilitation has gained significant attention in recent years due to its potential to enhance the recovery process for individuals with motor impairments resulting from neurological conditions and injuries. The main rehabilitation treatments rely on the repetitive execution of a movement of the upper-limb, guided by a therapist to prevent incorrect movements and to provide the necessary support. Many of the exercises performed by therapists can be modeled as a movement in SE(3) space (position and orientation). This movement itself is one-dimensional, as it can be modeled by a one-dimensional curve. To solve a similar problem, some approaches have been proposed in human-robot interaction (HRI) following virtual guides, but are either limited to specific types of curves (e.g. without orientation) or rely on linear control methods with non-intuitive parameters. To address these limitations and enable the use of these methods in physical rehabilitation, this paper extends Cartesian impedance control to splines, which we term path impedance control. It capitalizes on the intrinsic path geometry of end-effector robotic rehabilitation systems. The primary objective of this control algorithm is to emulate the sensation of maneuvering a physical object along a wire, akin to conventional exercise machines; and, in conjunction, provide an intuitive parametrization of rehabilitation exercises. We build on existing virtual guide control strategies using non-linear control and Lie Groups to generalize the control law to any one-parameter SE(3) curve.

Neural Style Transfer (NST) refers to a class of algorithms able to manipulate an element, most often images, to adopt the appearance or style of another one. Each element is defined as a combination of Content and Style: the Content can be conceptually defined as the “what” and the Style as the “how” of said element. In this context, we propose a custom NST framework for transferring a set of styles to the motion of a robotic manipulator, e.g., the same robotic task can be carried out in an “angry”, “happy”, “calm”, or “sad” way. An autoencoder architecture extracts and defines the Content and the Style of the target robot motions. A Twin Delayed Deep Deterministic Policy Gradient (TD3) network generates the robot control policy using the loss defined by the autoencoder. The proposed Neural Policy Style Transfer TD3 NPST3 alters the robot motion by introducing the trained style. Such an approach can be implemented either offline, for carrying out autonomous robot motions in dynamic environments, or online, for adapting at runtime the style of a teleoperated robot. The considered styles can be learned online from human demonstrations. We carried out an evaluation with human subjects enrolling 73 volunteers, asking them to recognize the style behind some representative robotic motions. Results show a good recognition rate, proving that it is possible to convey different styles to a robot using this approach.

Continuous Goal-Directed Actions (CGDA)is a robot imitation framework that encodes actions as the changes they produce on the environment. While it presents numerous advantages with respect to other robot imitation frameworks in terms of generalization and portability, final robot joint trajectories for the execution of actions are not necessarily encoded within the model. This is studied as an optimization problem, and the solution is computed through evolutionary algorithms in simulated environments. Evolutionary algorithms require a large number of evaluations, which had made the use of these algorithms in real world applications very challenging. This paper presents online evolutionary strategies, as a change of paradigm within CGDA execution. Online evolutionary strategies shift and merge motor execution into the planning loop. A concrete online evolutionary strategy, Online Evolved Trajectories (OET), is presented. OET drastically reduces computational times between motor executions, and enables working in real world dynamic environments and/or with human collaboration. Its performance has been measured against Full Trajectory Evolution (FTE)and Incrementally Evolved Trajectories (IET), obtaining the best overall results. Experimental evaluations are performed on the TEO full-sized humanoid robot with “paint” and “iron” actions that together involve vision, kinesthetic and force features.

As robotic systems become more popular in household environments, the complexity of required tasks also increases. In this work we focus on a domestic chore deemed dull by a majority of the population, the task of ironing. The presented algorithm improves on the limited number of previous works by joining 3D perception with force/torque sensing, with emphasis on finding a practical solution with a feasible implementation in a domestic setting. Our algorithm obtains a point cloud representation of the working environment. From this point cloud, the garment is segmented and a custom Wrinkleness Local Descriptor (WiLD) is computed to determine the location of the present wrinkles. Using this descriptor, the most suitable ironing path is computed and, based on it, the manipulation algorithm performs the force-controlled ironing operation. Experiments have been performed with a humanoid robot platform, proving that our algorithm is able to detect successfully wrinkles present in garments and iteratively reduce the wrinkleness using an unmodified iron.

Programming by demonstration (PbD) allows matching the kinematic movements of a robot with those of a human. The presented Continuous Goal-Directed Actions (CGDA) is able to additionally encode the effects of a demonstrated action, which are not encoded in PbD. CGDA allows generalization, recognition and execution of action effects on the environment. In addition to analyzing kinematic parameters (joint positions/velocities, etc.), CGDA focuses on changes produced on the object due to an action (spatial, color, shape, etc.). By tracking object features during action execution, we create a trajectory in an n-dimensional feature space that represents object temporal states. Discretized action repetitions provide us with a cloud of points. Action generalization is accomplished by extracting the average point of each sequential temporal interval of the point cloud. These points are interpolated using Radial Basis Functions, obtaining a generalized multidimensional object feature trajectory. Action recognition is performed by comparing the trajectory of a query sample with the generalizations. The trajectories discrepancy score is obtained by using Dynamic Time Warping (DTW). Robot joint trajectories for execution are computed in a simulator through evolutionary computation. Object features are extracted from sensors, and each evolutionary individual fitness is measured using DTW, comparing the simulated action with the generalization.

In this paper, we study how human-robot interaction can be beneficial on the Continuous Goal-Directed Actions (CGDA) framework. Specifically, a system for robot discovery of motor primitives from random human-guided movements has been developed. These guided motor primitives (GMP) are used as scaffolds to reproduce a goal-directed actions. CGDA encodes goals as the changes produced on object features (color, area, etc) due to actions. This paper focuses on using motor primitives extracted from human-guided random robot movements to execute these goal-directed actions. The human guides the robot joints in random movements, which are later divided in small segments. These segments are compared in terms of joint positions and selected to be diverse. To perform goal-directed actions, the robot must discover an adequate sequence of GMP. To discover these sequences we organize the primitives as a tree with incremental depths (where each node represents a primitive) and use a breadth-first search. In one of the experiments performed, the robot executes a task based on spatial object features. In the other experiment, the goal is to paint a wall by following a color feature trajectory.

An approach for adaptive shared control of an assistive manipulator is presented. A set of distributed collision and proximity sensors is used to aid in limiting collisions during direct control by the disabled user. Artificial neural networks adapt the use of the proximity sensors online, which limits movements in the direction of an obstacle before a collision occurs. The system learns by associating the different proximity sensors to the collision sensors where collisions are detected. This enables the user and the robot to adapt simultaneously and in real-time, with the objective of converging on a usage of the proximity sensors that increases performance for a given user, robot implementation and task-set. The system was tested in a controlled setting with a simulated 5 DOF assistive manipulator and showed promising reductions in the mean time on simplified manipulation tasks. It extends earlier work by showing that the approach can be applied to full multi-link manipulators.

This paper presents a robot imagination system that generates models of objects prior to their perception. This is achieved through a feature inference algorithm that enables computing the fusion of keywords which have never been presented to the robot together previously. In this sense, robot imagination is defined as the robot's capability of generating feature parameter values of unknown objects by generalizing characteristics from previously presented objects. The system is first trained with visual information paired with semantic object descriptions from which keywords are extracted. Each keyword creates an instance of the learnt object in an n-dimensional feature space. The core concept behind the robot imagination system presented in this paper is the use of statistically fit hyperplanes in the feature space to represent and simultaneously extend the meaning of grounded words. The inference algorithm allows to determine complete solutions in the feature space. Finally, evolutionary algorithms are used to return these numeric values to the real world, completing an inverse semantic process.

Increasingly disabled and elderly people with mobility problems want to live autonomously in their home environment. They are motivated to use robotic aids to perform tasks by themselves, avoiding permanent nurse or family assistant supervision. They must find means to rehabilitate their abilities to perform daily life activities (DLAs), such as eating, shaving, or drinking. These means may be provided by robotic aids that incorporate possibilities and methods to accomplish common tasks, aiding the user in recovery of partial or complete autonomy. Results are highly conditioned by the system's usability and potential. The developed portable assistive robot ASIBOT helps users perform most of these tasks in common living environments. Minimum adaptations are needed to provide the robot with mobility throughout the environment. The robot can autonomously climb from one surface to another, fixing itself to the best place to perform each task. When the robot is attached to its wheelchair, it can move along with it as a bundle. This paper presents the work performed with the ASIBOT in the area of rehabilitation robotics. First, a brief description of the ASIBOT system is given. A description of tests that have been performed with the robot and several impaired users is given. Insight into how these experiences have influenced our research efforts, especially, in home environments, is also included. A description of the test bed that has been developed to continue research on performing DLAs by the use of robotic aids, a kitchen environment, is given. Relevant conclusions are also included.