The perception of an object's surface is important for robotic applications enabling robust object manipulation. The level of accuracy in such a representation affects the outcome of the action planning, especially during tasks that require physical contact, e.g. grasping. In this paper, we propose a novel iterative method for 3D shape reconstruction consisting of two steps. At first, a mesh is fitted on data points acquired from the object's surface, based on a single primitive template. Subsequently, the mesh is properly adjusted to adequately represent local deformities. Moreover, a novel proactive tactile exploration strategy aims at minimizing the total uncertainty with the least number of contacts, while reducing the risk of contact failure in case the estimated surface differs significantly from the real one. The performance of the methodology is evaluated both in 3D simulation and on a real setup.

Robotic manipulation of deformable objects has drawn the attention of researchers over the past few years and is associated with a large spectrum of new application perspectives. In this article, we present an efficient integrated motion planning framework to effectively and accurately control a robotic manipulator executing interactive tasks on the surface of a deformable object. The proposed interactive motion planning framework is based on a mesh representation of the object, integrating three efficient preprocessing algorithmic steps, including visual object segmentation, finite element method deformation tracking, and local mesh parameterization. The use of barycentric coordinates, defined on the mesh triangles, enables the establishment of bijective transformations between the deformable part of an object surface and its planar (static and dynamic) parameterized mapping. By merging these spatial transformations with the preprocessing steps, in combination with an active stiffness scheme for robot manipulator control, we are able to achieve accurate and reactive motion planning of interactive trajectories, even under large and persistent visual occlusions (such as due to the presence of the robot in the visual scene). An extensive experimental evaluation study is presented, involving a robotic manipulator in interaction with a hemispherical model of controllable periodic active deformation, which permits precise ground truth derivation. Motion planning accuracy is evaluated in comparison with our previous direct vision-based approach, showing clearly superior performance of the proposed approach under all experimental conditions. The performance of the proposed framework is also further highlighted in tasks involving physical point tracking, interactive programming by human demonstration, as well as contact force regulation.

In this paper we introduce a novel technique that aims to control a two-module bio-inspired soft-robotic arm in order to qualitatively reproduce human demonstrations. The main idea behind the proposed methodology is based on the assumption that a complex trajectory can be derived from the composition and asynchronous activation of learned parameterizable simple movements constituting a knowledge base. The present work capitalises on recent research progress in Movement Primitive (MP) theory in order to initially build a library of Probabilistic MPs (ProMPs), and subsequently to compute on the fly their proper combination in the task space resulting in the requested trajectory. At the same time, a model learning method is assigned with the task to approximate the inverse kinematics, while a replanning procedure handles the sequential and/or parallel ProMPs' asynchronous activation. Taking advantage of the mapping at the primitive-level that the ProMP framework provides, the composition is transferred into the actuation space for execution. The proposed control architecture is experimentally evaluated on a real soft-robotic arm, where its capability to simplify the trajectory control task for robots of complex unmodeled dynamics is exhibited.

Online human leg tracking and gait analysis are crucial functionalities for mobility assistant robots, like intelligent walkers. Usually, such walkers are equipped with various sensors for the extraction of human-related features for adaptive human-robot interaction and assistance. We treat the gait detection problem jointly, presenting a novel method for detecting and recognizing gait features from 2D range data produced by a laser sensor mounted on a robotic walker. We propose an effective Convolutional Neural Network (CNN) as a powerful feature extractor for detecting the user’s leg centers in range data represented as occupancy grid maps. We couple the CNN with a Long Short Term Memory (LSTM) network for learning the legs’ motion temporal dynamics while walking, improving the prior detection, and providing better leg occlusion handling. Moreover, we perform gait analysis by recognizing gait phases over both legs by feeding the leg tracking output to a subsequent LSTM. Our proposed lightweight framework has been trained and tested on real patients-data. The presented experimental results show our method’s efficiency in providing accurate detections compared to state-of-the-art and application to an online system due to its high frequency, making it a competitive method for gait detection on robotic mobility assistants.

In this paper we introduce a novel technique that aims to dynamically control a two-module bio-inspired soft-robotic arm in order to qualitatively reproduce a path defined by sparse way-points. The main idea behind this work is based on the assumption that a complex trajectory may be derived as a combination of a discrete set of parameterizable simple movements, as suggested by Movement Primitive (MP) theory. Capitalising on recent advances in this field, the proposed controller uses a Probabilistic MP (ProMP) model which initially creates an abstract mapping in the primitive-level between the task and the actuation space, and subsequently guides the movement’s composition by exploiting its unique properties - conditioning and blending. At the same time, a learning-based adaptive controller updates the composition parameters by estimating the inverse kinematics of the robot, while an auxiliary process through replanning ensures that the trajectory complies with the new estimation. The learning architecture is evaluated on both a simulation model, and a real soft-robotic arm. The research findings show that the proposed methodology constitutes a novel approach that successfully manages to simplify the trajectory control task for robots of complex dynamics when high-precision is not required.

Elderly and mobility impaired people need special attention during bathing activities, since these tasks are demanding in body flexibility. Our aim is to build an assistive robotic bathing system, in order to increase the independence and safety of this procedure. Towards this end, the expertise of professional carers for bathing sequences and appropriate motions have to be adopted, in order to achieve natural, physical human - robot interaction. In this paper a Navigation Function (NF) approach is proposed in order to reproduce the way an expert clinical carer executes the bathing activities by means of construction repulsive potential fields ("virtual obstacles") for an assistive bath robot. The produced vector field, constructed based on the demonstration procedure, is used for real-time motion behavior planning tasks, which exploits the visual information from Depth sensors and the advantages of the NF approach, to estimate the reference pose for the end- effector of the assistive robotic system. The proposed method guarantees globally asymptotic convergence to the learned from demonstration washing motion, within the deformable and moving body-part limits, while in addition, restricted areas on the body surface are avoided. The proposed method is evaluated using real experimental data, obtained from human subjects during pouring water task demonstration.

In this paper we introduce a novel technique that aims to dynamically control a modular bio-inspired soft-robotic arm in order to perform cyclic rhythmic patterns. Oscillatory signals are produced at the actuator's level by a central pattern generator (CPG), resulting in the generation of a periodic motion by the robot's end-effector. The proposed controller is based on a model-free neurodynamic scheme and is assigned with the task of training a policy that computes the parameters of the CPG model which generates a trajectory with desired features. The proposed methodology is first evaluated with a simulation model, which successfully reproduces the trained targets. Then experiments are also conducted using the real robot. Both procedures validate the efficiency of the learning architecture to successfully complete these tasks.

The integration of visual perception to robotic systems is a key research area in recent years. Advances in modern computer vision techniques along with the development of faster and more accurate visual sensors led to the emergence of new methods for robotic visual perception [1]. One area of research is the development of robotic assistive vision for Human-Robot Interaction (HRI) systems [2], [3]. The rapid increase of people with special needs, such as the elderly population, and the simultaneous reduction of personal care staff, reinforce the need for robotic assistants [4], [5]. There are many challenges in this area including the familiarity of these users with new technologies and the domain specific datasets, which are required for training user oriented models. Nowadays, modern assistive and social human-robot interaction requires the multimodal communication with speech, gestures and human movements so as to enhance the classic interaction with only spoken commands.

In this work, we present a novel framework for on-line human gait stability prediction of the elderly users of an intelligent robotic rollator using Long Short Term Memory (LSTM) networks, fusing multimodal RGB-D and Laser Range Finder (LRF) data from non-wearable sensors. A Deep Learning (DL) based approach is used for the upper body pose estimation. The detected pose is used for estimating the body Center of Mass (CoM) using Unscented Kalman Filter (UKF). An Augmented Gait State Estimation framework exploits the LRF data to estimate the legs' positions and the respective gait phase. These estimates are the inputs of an encoder-decoder sequence to sequence model which predicts the gait stability state as Safe or Fall Risk walking. It is validated with data from real patients, by exploring different network architectures, hyperparameter settings and by comparing the proposed method with other baselines. The presented LSTM-based human gait stability predictor is shown to provide robust predictions of the human stability state, and thus has the potential to be integrated into a general user-adaptive control architecture as a fall-risk alarm.

In this work, we tackle the problem of child engagement estimation while children freely interact with a robot in a friendly, room-like environment. We propose a deep learning-based multi-view solution that takes advantage of recent developments in human pose detection. We extract the child's pose from different RGB-D cameras placed regularly in the room, fuse the results and feed them to a deep Neural Network (NN) trained for classifying engagement levels. The deep network contains a recurrent layer, in order to exploit the rich temporal information contained in the pose data. The resulting method outperforms a number of baseline classifiers and provides a promising tool for better automatic understanding of a child's attitude, interest and attention while cooperating with a robot. The goal is to integrate this model in next-generation social robots as an attention monitoring tool during various Child Robot Interaction (CRI) tasks both for Typically Developed (TD) children and children affected by autism (ASD).

We explore new aspects of assistive living on smart human-robot interaction (HRI) that involve automatic recognition and online validation of speech and gestures in a natural interface, providing social features for HRI. We introduce a whole framework and resources of a real-life scenario for elderly subjects supported by an assistive bathing robot, addressing health and hygiene care issues. We contribute a new dataset and a suite of tools used for data acquisition and a state-of-the-art pipeline for multimodal learning within the framework of the I -Support bathing robot, with emphasis on audio and RGB- D visual streams. We consider privacy issues by evaluating the depth visual stream along with the RGB, using Kinect sensors. The audio-gestural recognition task on this new dataset yields up to 84.5%, while the online validation of the I-Support system on elderly users accomplishes up to 84% when the two modalities are fused together. The results are promising enough to support further research in the area of multimodal recognition for assistive social HRI, considering the difficulties of the specific task.

In this paper we describe a control strategy for a user-adaptive human-robot system for an intelligent robotic Mobility Assistive Device (MAD)using raw data from a single laser-range-finder (LRF)mounted on the MAD and scanning the walking area. The proposed control architecture consists of three modules. In the first module, a previously proposed methodology (termed IMM-PDA-PF)delivers the augmented human state estimation of the user by providing robust leg tracking and on-line estimation of the human gait phases. This information is processed at the next module for providing the pathological gait parametrization and characterization, by computing specific gait parameters for each gait cycle. These gait parameters form the feature vector that classifies the user in a certain class related to risk of fall. Those are of particular significance to the system, since the gait parameters and the respective class are used in the third module, i.e. the human-robot formation controller, in order to adapt the desired formation of the human-robot system, by selecting the appropriate control variables. The experimental evaluation comprises gait data from real patients, and demonstrates the stability of the human-robot formation control, indicating the importance of incorporating an on-line gait characterization of the user, using non-wearable and non-invasive methods, in the context of a robotic MAD.

This work examines how and to what benefit an autonomous humanoid robot can supervise a child in an object assembly task. In order to understand the child's actions, a novel 3D object tracking algorithm for RGB-D data is employed. The tracker consists of two stages: the first performs a tracking-by-detection scheme on the color stream, to locate the objects on the image plane, while the second uses a particle filter that operates on the depth data stream to refine the first stage output and infer the objects' rotations. Given the six degrees-of-freedom of the assembly part poses, the system is able to recognize which connections have been completed at any given time. This information is then used to select an appropriate verbal or gestural response for the robot. Experimental results show that (a) the tracking algorithm is accurate, fast and robust to severe occlusions and fast movements, (b) the proposed method of assembly state estimation is indeed effective, and (c) the resulting Child-Robot Interaction scenario is educational and enjoyable for the children involved.

Using robots as therapeutic or educational tools for children with autism requires robots to be able to adapt their behavior specifically for each child with whom they interact. In particular, some children may like to be looked into the eyes by the robot while some may not. Some may like a robot with an extroverted behavior while others may prefer a more introverted behavior. Here we present an algorithm to adapt the robot's expressivity parameters of action (mutual gaze duration, hand movement expressivity) in an online manner during the interaction. The reward signal used for learning is based on an estimation of the child's mutual engagement with the robot, measured through non-verbal cues such as the child's gaze and distance from the robot. We first present a pilot joint attention task where children with autism interact with a robot whose level of expressivity is pre-determined to progressively increase, and show results suggesting the need for online adaptation of expressivity. We then present the proposed learning algorithm and some promising simulations in the same task. Altogether, these results suggest a way to enable robot learning based on non-verbal cues and to cope with the high degree of nonstationarities that can occur during interaction with children.

Tracking human gait accurately and robustly constitutes a key factor for a smart robotic walker, aiming to provide assistance to patients with different mobility impairment. A context-aware assistive robot needs constant knowledge of the user's kinematic state to assess the gait status and adjust its movement properly to provide optimal assistance. In this work, we experimentally validate the performance of two gait tracking algorithms using data from elderly patients; the first algorithm employs a Kalman Filter (KF), while the second one tracks the user legs separately using two probabilistically associated Particle Filters (PFs). The algorithms are compared according to their accuracy and robustness, using data captured from real experiments, where elderly subjects performed specific walking scenarios with physical assistance from a prototype Robotic Rollator. Sensorial data were provided by a laser rangefinder mounted on the robotic platform recording the movement of the user's legs. The accuracy of the proposed algorithms is analysed and validated with respect to ground truth data provided by a Motion Capture system tracking a set of visual markers worn by the patients. The robustness of the two tracking algorithms is also analysed comparatively in a complex maneuvering scenario. Current experimental findings demonstrate the superior performance of the PFs in difficult cases of occlusions and clutter, where KF tracking often fails.

When designing a user-friendly Mobility Assistive Device (MAD) for mobility constrained people, it is important to take into account the diverse spectrum of disabilities, which results to completely different needs to be covered by the MAD for each specific user. An intelligent adaptive behavior is necessary. In this work we present experimental results, using an in house developed methodology for assessing the gait of users with different mobility status while interacting with a robotic MAD. We use data from a laser scanner, mounted on the MAD to track the legs using Particle Filters and Probabilistic Data Association (PDA-PF). The legs' states are fed to an HMM-based pathological gait cycle recognition system to compute in real-time the gait parameters that are crucial for the mobility status characterization of the user. We aim to show that a gait assessment system would be an important feedback for an intelligent MAD. Thus, we use this system to compare the gaits of the subjects using two different control settings of the MAD and we experimentally validate the ability of our system to recognize the impact of the control designs on the users' walking performance. The results demonstrate that a generic control scheme does not meet every patient's needs, and therefore, an Adaptive Context-Aware MAD (ACA MAD), that can understand the specific needs of the user, is important for enhancing the human-robot physical interaction.

Elderly people have particular needs in performing bathing activities, since these tasks require body flexibility. Our aim is to build an assistive robotic bath system, in order to increase the independence and safety of this procedure. Towards this end, the expertise of professional carers for bathing sequences and appropriate motions has to be adopted, in order to achieve natural, physical human - robot interaction. In this paper, a real-time end-effector motion planning method for an assistive bath robot, using on-line Point-Cloud information, is proposed. The visual feedback obtained from Kinect depth sensor is employed to adapt suitable washing paths to the user's body part motion and deformable surface. We make use of a navigation function-based controller, with guarantied globally uniformly asymptotic stability, and bijective transformations for the adaptation of the paths. Experiments were conducted with a rigid rectangular object for validation purposes, while a female subject took part to the experiment in order to evaluate and demonstrate the basic concepts of the proposed methodology.

For a robotic walker designed to assist mobility constrained people, it is important to take into account the different spectrum of pathological walking patterns, which result into completely different needs to be covered for each specific user. For a deployable intelligent assistant robot it is necessary to have a precise gait analysis system, providing real-time monitoring of the user and extracting specific gait parameters, which are associated with the rehabilitation progress and the risk of fall. In this paper, we present a completely non-invasive framework for the on-line analysis of pathological human gait and the recognition of specific gait phases and events. The performance of this gait analysis system is assessed, in particular, as related to the estimation of double support phases, which are typically difficult to extract reliably, especially when applying non-wearable and non-intrusive technologies. Furthermore, the duration of double support phases constitutes an important gait parameter and a critical indicator in pathological gait patterns. The performance of this framework is assessed using real data collected from an ensemble of elderly persons with different pathologies. The estimated gait parameters are experimentally validated using ground truth data provided by a Motion Capture system. The results obtained and presented in this paper demonstrate that the proposed human data analysis (modeling, learning and inference) framework has the potential to support efficient detection and classification of specific walking pathologies, as needed to empower a cognitive robotic mobility-assistance device with user-adaptive and context-aware functionalities.

This paper proposes a model-free learning scheme for the developmental acquisition of robot kinematic control and dexterous manipulation skills. The approach is based on a nested-hierarchical multi-agent architecture that intuitively encapsulates the topology of robot kinematic chains, where the activity of each independent degree-of-freedom (DOF) is finally mapped onto a distinct agent. Each one of those agents progressively evolves a local kinematic control strategy in a game-theoretic sense, that is, based on a partial (local) view of the whole system topology, which is incrementally updated through a recursive communication process according to the nested-hierarchical topology. Learning is thus approached not through demonstration and training but through an autonomous self-exploration process. A fuzzy reinforcement learning scheme is employed within each agent to enable efficient exploration in a continuous state–action domain. This paper constitutes in fact a proof of concept, demonstrating that global dexterous manipulation skills can indeed evolve through such a distributed iterative learning of local agent sensorimotor mappings. The main motivation behind the development of such an incremental multi-agent topology is to enhance system modularity, to facilitate extensibility to more complex problem domains and to improve robustness with respect to structural variations including unpredictable internal failures. These attributes of the proposed system are assessed in this paper through numerical experiments in different robot manipulation task scenarios, involving both single and multi-robot kinematic chains. The generalisation capacity of the learning scheme is experimentally assessed and robustness properties of the multi-agent system are also evaluated with respect to unpredictable variations in the kinematic topology. Furthermore, these numerical experiments demonstrate the scalability properties of the proposed nested-hierarchical architecture, where new agents can be recursively added in the hierarchy to encapsulate individual active DOFs. The results presented in this paper demonstrate the feasibility of such a distributed multi-agent control framework, showing that the solutions which emerge are plausible and near-optimal. Numerical efficiency and computational cost issues are also discussed.

The precise analysis of a patient's or an elderly person's walking pattern is very important for an effective intelligent active mobility assistance robot. This walking pattern can be described by a cyclic motion, which can be modeled using the consecutive gait phases. In this paper, we present a completely non-invasive framework for analyzing and recognizing a pathological human walking gait pattern. Our framework utilizes a laser range finder sensor to detect and track the human legs, and an appropriately synthesized Hidden Markov Model (HMM) for state estimation, and recognition of the gait patterns. We demonstrate the applicability of this setup using real data, collected from an ensemble of different elderly persons with a number of pathologies. The results presented in this paper demonstrate that the proposed human data analysis scheme has the potential to provide the necessary methodological (modeling, inference, and learning) framework for a cognitive behavior-based robot control system. More specifically, the proposed framework has the potential to be used for the classification of specific walking pathologies, which is needed for the development of a context-aware robot mobility assistant.

For an effective intelligent active mobility assistance robot, the walking pattern of a patient or an elderly person has to be analyzed precisely. A well-known fact is that the walking patterns are gaits, that is, cyclic patterns with several consecutive phases. These cyclic motions can be modeled using the consecutive gait phases. In this paper, we present a completely non-invasive framework for analyzing a normal human walking gait pattern. Our framework utilizes a laser range finder sensor to collect the data, a combination of filters to preprocess these data, and an appropriately synthesized Hidden Markov Model (HMM) for state estimation, and recognition of the gait data. We demonstrate the applicability of this setup using real data, collected from an ensemble of different persons. The results presented in this paper demonstrate that the proposed human data analysis scheme has the potential to provide the necessary methodological (modeling, inference, and learning) framework for a cognitive behavior-based robot control system. More specifically, the proposed framework has the potential to be used for the recognition of abnormal gait patterns and the subsequent classification of specific walking pathologies, which is needed for the development of a context-aware robot mobility assistant.

This paper presents a shared control approach for motion compensation in robotic beating heart surgery. Motion compensation consists of three main tasks; motion synchronization, image stabilization and shared control. The paper discusses a unifying framework under which the three tasks combine seamlessly. In this work, the planar 1-manifold case is considered, where a strip-wise affine map is performed to achieve image stabilization onto a canonical space, where shared control emerges naturally. A prototype teleoperation system is also described, implementing the algorithms. Experiments were performed with medically trained users, and the positive effect of motion compensation is analyzed.

In this paper a novel approach for 3D environment mapping using registered robot poses is presented. The proposed algorithm focuses on improving the quality of robot generated 3D maps by incorporating the uncertainty of 3D points and propagating it into the normal vectors of surfaces. The uncertainty of normal vectors is an indicator of the quality of the detected surface. A controlled random search algorithm is applied to optimize a non-convex function of uncertain normal vectors and number of clusters in order to find the optimal threshold parameter for the segmentation process. This approach leads to an improved cluster coherence and thus better maps.

This paper proposes a model-mediated telemanipulation scheme, focusing on the kinaesthetic perception of specific geometric properties of the remote environment in the presence of time delay. The basic idea is inspired from previous work on impedance-reflection teleoperation, which is here extended to incorporate the construction of a two-dimensional local geometric model. This model incorporates the local curvature of the remote environment, estimated online using a recursive least squares (RLS) method, which is then used to reconstruct a virtual surface model at the master site for haptic display. A series of experiments has been conducted, where each subject manipulated the haptic master to kinaes-thetically explore the surface of a remote (virtual) environment. The analysis of the obtained experimental results, in terms of telehaptic discrimination of curvature, shows the effectiveness of the proposed model-mediated scheme at mitigating some of the adverse effects of time delay in the communication loop.

In present days, teleoperation is used in many challenging applications where the tasks to be accomplished from a distance are very complex and require accurate and reliable reproduction of the haptic sensations involved. The main factor that can cause a certain degradation of the quality of teleoperation is the presence of time delays. An effective way of alleviating the consequences of time-delays is the use of an adaptive impedance reflection teleoperation scheme, aiming to reconstruct at the master site a local model of the impedance of the remote environment. The goal of this paper is to show the effectiveness of such a controller via experiments that involve a real remote environment. In these experiments, a forced-choice procedure has been used, where each subject is presented in every trial with two spring fields (remotely located and telehaptically perceived) and is asked to identify the stiffer. The proposed adaptive teleoperation control scheme is compared to a typical direct force-reflection telemanipulation, in the presence of an emulated time delay of 100 msec. Experimental results show the superior performance of the proposed adaptive impedance reflection scheme, which, as opposed to classical direct teleoperation, seems to maintain the thresholds of human haptic perception close to the ones obtained when no time delay is present in the bilateral communication and control loop.

This paper presents results achieved in the frames of a national research project (titled ldquoDIANOEMArdquo), where visual analysis and sign recognition techniques have been explored on Greek Sign Language (GSL) data. Besides GSL modelling, the aim was to develop a pilot application for teleoperating a mobile robot using natural hand signs. A small vocabulary of hand signs has been designed to enable desktopbased teleoperation at a high-level of supervisory telerobotic control. Real-time visual recognition of the hand images is performed by training a multi-layer perceptron (MLP) neural network. Various shape descriptors of the segmented hand posture images have been explored as inputs to the MLP network. These include Fourier shape descriptors on the contour of the segmented hand sign images, moments, compactness, eccentricity, and histogram of the curvature. We have examined which of these shape descriptors are best suited for real-time recognition of hand signs, in relation to the number and choice of hand postures, in order to achieve maximum recognition performance. The hand-sign recognizer has been integrated in a graphical user interface, and has been implemented with success on a pilot application for real-time desktop-based gestural teleoperation of a mobile robot vehicle.

This paper proposes the application of an adaptive impedance control scheme to alleviate some of the problems associated with the presence of time delays in a haptic teleoperation system. Continuous on-line estimation of the remote environment's impedance is performed, and is then used as a local model for haptic display control. Lyapunov stability of the proposed impedance adaptation law is demonstrated. A series of experiments is performed to evaluate the performance of this teleoperation control scheme. Two performance measures are defined to assess transparency and stability of the teleoperator. Simulation results show the superior performance of the proposed adaptive scheme, with respect to direct teleoperation, particularly in terms of increasing the stability margin and of significantly ameliorating transparency in the presence of large time delays. Experimental results, using a phantom omni as the haptic master device, support this conclusion.

Simultaneous Localization and Mapping in dynamic environments is an open issue in the field of robotics. Traditionally, the related approaches assume that the environment remains static during the robot's exploration phase. In this work, we overcome this assumption and propose an algorithm that exploits the dynamic nature of the environment during robot exploration so as to improve the localization process. We use a Histogram Grid to store all the past occupancy values of every cell and thus to select the most probable pose of the robot based on the occupancy evolution. Experiments on a simulated robot indicate the effectiveness of the proposed approach.

With the spread of low-cost haptic devices, haptic interfaces appear in many areas in the field of robotics. Recently, haptic devices have been used in the field of mobile robot teleoperation, where mobile robots operate in unknown and dangerous environments performing particular tasks. Haptic feedback is shown to improve operator perception of the environment without, however, improving exploration time. In this paper, we present a haptic interface that is used to teleoperate a mobile robot in exploring polygonal environments. The proposed visuo-haptic interface is found to improve navigation time and operator perception of the remote environment. The human-operator can simultaneously select two different commands, the first one being set as "active" motion command, while the second one is set as a "guarded" motion type of navigation command. The user can feel a haptic equivalent for both types of teleguidance motion commands, and can also observe in real-time the sequential creation of the remote environment map. Comparative evaluation experiments show that the proposed system makes the task of remote navigation of unknown environments easier

One of the key requirements for a Virtual Reality system is the multimodal, real-time interaction between the human operator and a computer simulated and animated environment. This paper investigates problems related particularly to the haptic interaction between the human operator and a virtual environment. The work presented here focuses on two issues: 1) the synthesis of whole-hand kinesthetic feedback, based on the application of forces (torques) on individual phalanges (joints) of the human hand, and 2) the experimental evaluation of this haptic feedback system, in terms of human haptic perception of virtual physical properties (such as the weight of a virtual manipulated object), using psychophysical methods. The proposed kinesthetic feedback methodology is based on the solution of a generalized force distribution problem for the human hand during virtual manipulation tasks. The solution is computationally efficient and has been experimentally implemented using an exoskeleton force-feedback glove. A series of experiments is reported concerning the perception of weight of manipulated virtual objects and the obtained results demonstrate the feasibility of the concept. Issues related to the use of sensory substitution techniques for the application of haptic feedback on the human hand are also discussed.

Segmentation of soil section images is an important task for automating the measurement of the grains' properties as well as for detecting and recognizing objects in the soil, important for its bioecological quality. We apply several types of morphological systems to watershed-based segmentation of soil section images. We use efficient connected operators such as reconstruction open-closing and area open-closing as well as some relatively new operators, the levelings, for image denoising, simplification and feature/marker extraction. Further, we introduce an improvement of the reconstruction operators used in segmentation, based on a generalized multiscale connectivity analysis.

This paper focuses on the problem of force-feedback for the human-operator hand when manipulating virtual objects. We propose a method for the computation of feedback-forces that have to be applied on each individual phalanx and finger of the human hand in order to display pertinent, kinesthetic information about static or dynamic characteristics of objects present in the virtual scene. External forces and moments of the manipulated virtual objects heave to be mapped on the contact-forces space of the virtual grasp. The method is based on the solution of a nonlinear programming problem, formulated by performing a static analysis of a general, multiple contact points virtual grasp. A methodology for modelling interactions within a virtual environment, and performing realistic grasping and manipulation, is also presented.

This paper discusses the development of new teleoperator systems. While many innovations during the last decade made teleoperation technology progress, some severe well known lacks that we enumerate still persist. With respect to some attractive solutions proposed for coping with these problems we designed a bilateral control scheme based on what we called the hidden robot concept. The teleoperator achieves tasks manually in a natural way within a virtual environment (VE). Thanks to suitable bilateral transformations, the virtual tasks are being reproduced by any slave robot within the remote site. Mainly task based, our approach is not considered like a high level task knowledge based control. Rather, we consider it like a more refined shared autonomy control with a high level abstraction interface. Three main components are developed: (i) supervision loop, (ii) bilateral transformation layer, (iii) execution loop. The approach has been validated experimentally and preliminary results as well as further work are discussed.

In this paper we propose an adaptive impedance control scheme consisting of two stages. The first stage performs an online estimation of the robot parameters imposing the desired mechanical impedance. It constitutes the kernel of the control system and remains active during the complete, free or constrained, motion of the robot. A simple algorithm for the numerical computation of the defined impedance error is presented where only the available feedback information is used. The imprecision in the parameters of the environment (stiffness, positioning) is compensated by the second stage of adaptation. This one constitutes an external force control loop closed around the internal nonlinear impedance controller. Simulation results obtained for a single leg of a pneumatic driven, quadruped robot show the effectiveness of the proposed control scheme in case of considerable uncertainty both in the robot and ground parameters.