Robotic manipulation of deformable objects has drawn the attention of researchers over the past few years and is associated with a large spectrum of new application perspectives. In this article, we present an efficient integrated motion planning framework to effectively and accurately control a robotic manipulator executing interactive tasks on the surface of a deformable object. The proposed interactive motion planning framework is based on a mesh representation of the object, integrating three efficient preprocessing algorithmic steps, including visual object segmentation, finite element method deformation tracking, and local mesh parameterization. The use of barycentric coordinates, defined on the mesh triangles, enables the establishment of bijective transformations between the deformable part of an object surface and its planar (static and dynamic) parameterized mapping. By merging these spatial transformations with the preprocessing steps, in combination with an active stiffness scheme for robot manipulator control, we are able to achieve accurate and reactive motion planning of interactive trajectories, even under large and persistent visual occlusions (such as due to the presence of the robot in the visual scene). An extensive experimental evaluation study is presented, involving a robotic manipulator in interaction with a hemispherical model of controllable periodic active deformation, which permits precise ground truth derivation. Motion planning accuracy is evaluated in comparison with our previous direct vision-based approach, showing clearly superior performance of the proposed approach under all experimental conditions. The performance of the proposed framework is also further highlighted in tasks involving physical point tracking, interactive programming by human demonstration, as well as contact force regulation.

In this paper we introduce a novel technique that aims to control a two-module bio-inspired soft-robotic arm in order to qualitatively reproduce human demonstrations. The main idea behind the proposed methodology is based on the assumption that a complex trajectory can be derived from the composition and asynchronous activation of learned parameterizable simple movements constituting a knowledge base. The present work capitalises on recent research progress in Movement Primitive (MP) theory in order to initially build a library of Probabilistic MPs (ProMPs), and subsequently to compute on the fly their proper combination in the task space resulting in the requested trajectory. At the same time, a model learning method is assigned with the task to approximate the inverse kinematics, while a replanning procedure handles the sequential and/or parallel ProMPs' asynchronous activation. Taking advantage of the mapping at the primitive-level that the ProMP framework provides, the composition is transferred into the actuation space for execution. The proposed control architecture is experimentally evaluated on a real soft-robotic arm, where its capability to simplify the trajectory control task for robots of complex unmodeled dynamics is exhibited.

Online human leg tracking and gait analysis are crucial functionalities for mobility assistant robots, like intelligent walkers. Usually, such walkers are equipped with various sensors for the extraction of human-related features for adaptive human-robot interaction and assistance. We treat the gait detection problem jointly, presenting a novel method for detecting and recognizing gait features from 2D range data produced by a laser sensor mounted on a robotic walker. We propose an effective Convolutional Neural Network (CNN) as a powerful feature extractor for detecting the user’s leg centers in range data represented as occupancy grid maps. We couple the CNN with a Long Short Term Memory (LSTM) network for learning the legs’ motion temporal dynamics while walking, improving the prior detection, and providing better leg occlusion handling. Moreover, we perform gait analysis by recognizing gait phases over both legs by feeding the leg tracking output to a subsequent LSTM. Our proposed lightweight framework has been trained and tested on real patients-data. The presented experimental results show our method’s efficiency in providing accurate detections compared to state-of-the-art and application to an online system due to its high frequency, making it a competitive method for gait detection on robotic mobility assistants.

In this paper we introduce a novel technique that aims to dynamically control a two-module bio-inspired soft-robotic arm in order to qualitatively reproduce a path defined by sparse way-points. The main idea behind this work is based on the assumption that a complex trajectory may be derived as a combination of a discrete set of parameterizable simple movements, as suggested by Movement Primitive (MP) theory. Capitalising on recent advances in this field, the proposed controller uses a Probabilistic MP (ProMP) model which initially creates an abstract mapping in the primitive-level between the task and the actuation space, and subsequently guides the movement’s composition by exploiting its unique properties - conditioning and blending. At the same time, a learning-based adaptive controller updates the composition parameters by estimating the inverse kinematics of the robot, while an auxiliary process through replanning ensures that the trajectory complies with the new estimation. The learning architecture is evaluated on both a simulation model, and a real soft-robotic arm. The research findings show that the proposed methodology constitutes a novel approach that successfully manages to simplify the trajectory control task for robots of complex dynamics when high-precision is not required.

Elderly and mobility impaired people need special attention during bathing activities, since these tasks are demanding in body flexibility. Our aim is to build an assistive robotic bathing system, in order to increase the independence and safety of this procedure. Towards this end, the expertise of professional carers for bathing sequences and appropriate motions have to be adopted, in order to achieve natural, physical human - robot interaction. In this paper a Navigation Function (NF) approach is proposed in order to reproduce the way an expert clinical carer executes the bathing activities by means of construction repulsive potential fields ("virtual obstacles") for an assistive bath robot. The produced vector field, constructed based on the demonstration procedure, is used for real-time motion behavior planning tasks, which exploits the visual information from Depth sensors and the advantages of the NF approach, to estimate the reference pose for the end- effector of the assistive robotic system. The proposed method guarantees globally asymptotic convergence to the learned from demonstration washing motion, within the deformable and moving body-part limits, while in addition, restricted areas on the body surface are avoided. The proposed method is evaluated using real experimental data, obtained from human subjects during pouring water task demonstration.

We explore new aspects of assistive living on smart human-robot interaction (HRI) that involve automatic recognition and online validation of speech and gestures in a natural interface, providing social features for HRI. We introduce a whole framework and resources of a real-life scenario for elderly subjects supported by an assistive bathing robot, addressing health and hygiene care issues. We contribute a new dataset and a suite of tools used for data acquisition and a state-of-the-art pipeline for multimodal learning within the framework of the I -Support bathing robot, with emphasis on audio and RGB- D visual streams. We consider privacy issues by evaluating the depth visual stream along with the RGB, using Kinect sensors. The audio-gestural recognition task on this new dataset yields up to 84.5%, while the online validation of the I-Support system on elderly users accomplishes up to 84% when the two modalities are fused together. The results are promising enough to support further research in the area of multimodal recognition for assistive social HRI, considering the difficulties of the specific task.

Elderly people have particular needs in performing bathing activities, since these tasks require body flexibility. Our aim is to build an assistive robotic bath system, in order to increase the independence and safety of this procedure. Towards this end, the expertise of professional carers for bathing sequences and appropriate motions has to be adopted, in order to achieve natural, physical human - robot interaction. In this paper, a real-time end-effector motion planning method for an assistive bath robot, using on-line Point-Cloud information, is proposed. The visual feedback obtained from Kinect depth sensor is employed to adapt suitable washing paths to the user's body part motion and deformable surface. We make use of a navigation function-based controller, with guarantied globally uniformly asymptotic stability, and bijective transformations for the adaptation of the paths. Experiments were conducted with a rigid rectangular object for validation purposes, while a female subject took part to the experiment in order to evaluate and demonstrate the basic concepts of the proposed methodology.