Estimating a person’s readiness to engage is a critical prerequisite for achieving natural interactions with machines. In this work, we explore a skeleton-based approach for estimating interaction readiness from human motion. A two-stream spatio-temporal graph convolutional network is applied as backbone. We propose two design changes to the backbone: Local Dense Connection (LDC), which enhances the flow of multi-scale features, and Cross-Stream Attention (CSA) module, allowing it to effectively relate joint and bone features. Rather than classifying actions directly, a probabilistic aggregation strategy is introduced to generate a scalar measure of interaction readiness, which helps the model generalize better to real-world scenes. Experiment on the processed NTU-RGB+D 120 dataset demonstrates the proposed method achieves 82.52% top-1 accuracy, outperforming backbone model. Moreover, experiment on real-world data achieves ROC-AUC = 0.9687 in realistic conditions, indicating the robustness and generalization ability of the proposed method with lightweight parameters (8.30 M). While relatively lightweight, the method offers a practical solution for scenarios that require fast, interpretable estimation such as human–robot interaction settings.

Exoskeletons are wearable robotic devices that surround the anatomy of the user to work in tandem with it. Depending on their structure, exoskeletons can be classified as rigid or flexible. The structure of flexible exoskeletons is made of soft materials, such as fabrics, which adapt to user motion. Therefore, these devices are prone to becoming misaligned with the user, due to improper fitting and slipping of the exoskeleton components on the user body. This article describes a cable-driven exosuit, calledLUXBIT, that favors its anatomical adaption to the user by arranging the fabric fibers and sewing patterns to transfer the mobilizing forces. This prototype integrates a novel deformable mechanism that promotes the natural lifting of the arm.LUXBITis intended for bimanual assistance in daily living, being equipped with a backpack to this end. The results analyzed in this article show thatLUXBITreduces muscle activity in the upper limbs’ flexion by ratios of up to 13.17% and allows the user to hold tiring postures for 62.91% longer.

In order to perform safe robotic interventions in harsh environments it is necessary to help the robotic operator with a Human-Robot Interface that provides multimodal interactions, from low level interaction methods to bilateral teleoperation with force feedback. These interaction modalities, though, rely purely on the operator's skills. With the objective of providing a safer system, higher-level applications can be integrated in the interface in order to provide some help to the operator, without relying uniquely on his/her capacities. This paper presents a novel object recognition and tracking system which runs in real-time on the robot while the operator is operating it. The tracking system enters in the teleoperation loop and helps the operator to achieve the requested goals. The system is optimized to track featureless objects such as metallic plates, metallic connectors and monochromatic objects. Moreover, the algorithm provides improvements with respect to previous tracking experiments, including depth estimation in order to better interact with the velocity control of the robotic arm when approaching the target, as well as high reliability with partial occlusions. This vision-based control system is used in real interventions in hazardous environments, in order to track and manipulate metallic parts of scientific and engineering machines, giving a performance success over 95%, and reaching the 100% under the remote human supervision.

Despite the substantial progression of autonomous driving systems, their application is often limited e.g. due to safety margins which can be caused by uncertainties in the environment reconstruction. Then, via teleoperation as a fallback solution, a human-in-the-loop can be introduced as the main decision maker. However, high delay in the communication channel distorts the performance of direct force feedback teleoperation for example in space or disaster scenarios. On the other hand, model-mediated teleoperation can provide instantaneous and even predictive force feedback to the user, but the performance is limited due to state mismatches, incomplete models, model errors and the modeling challenges of complex wheel-ground contacts. Therefore, in this paper we introduce the concept of extended model-mediated teleoperation with a car like interface for mobile robots by fusing local fictitious and remote force feedback, which can be measured, computed or fictitious. We provide a method to guarantee stability of the extended model-mediated teleoperation (involving time delay, multilateral coupling, fictitious force feedback and permanent updates of the local model) based on the passivity theorem. The benefits of the approach are highlighted by human-in-the-loop experiments with a wheeled mobile robot.

Multiple robotic agents, autonomous or teleoperated, can be employed to synergise and cooperate to achieve a common objective more effectively. Tasks using robotic manipulators can be eased and improved in terms of reliability, adaptability and ergonomics via robot cooperation. In spite of visual and haptic feedback, cooperative telemanipulation of multiple robots by distant operators can still be challenging due to practical limitations in synchronisation and supervision. This paper presents a new control approach for haptic intention augmentation between two human operators handling objects via teleoperation in a cooperative manner. The force feedback to each operator is enhanced by information on the motion intention of the other operator observed by a force sensor at the input devices. Besides on-ground experiments, an experiment is presented that involves the cooperative teleoperation of an on-ground robot by a cosmonaut on the International Space Station and another distant operator on ground.

This paper describes the experiences using remote laboratories for education and research in the field of Control Engineering. The use of remote laboratories for education in subjects of control is increasingly becoming a resorted method by the universities in order to offer a flexible service in schedules with greater and better operation of available resources. Nevertheless, for research activities, remote laboratories are not widely used. The aim of this contribution is thereby to apply the experience of remote laboratories in research applications in order to share complex equipments among different researchers. Some experiments are carried out to demonstrate the effectiveness of using remote laboratories in research experiments related to robotic system. The results of the implementation of remote experimentations to control a 3-DOF parallel robot by using Distance Laboratory System (SLD) are exposed. The performance of the system is evaluated by the possibilities and functionality of the proposed remote laboratory platform.

This paper describes the experiences using remote laboratories for thorough analysis of a thermal system, including disturbances. Remote laboratories for education in subjects of control, is a common resorted method, used by universities. This method is applied to offer a flexible service in schedules so as to obtain greater and better results of available resources. Remote laboratories have been used for controlling physical devices remotely. Furthermore, remote labs have been used for transfer function identification of real equipment. Nevertheless, remote analyses of disturbances have not been done. The aim of this contribution is thereby to apply the experience of remote laboratories in the study of disturbances. Some experiments are carried out to demonstrate the effectiveness in using remote laboratories for complete analysis of a thermal system. Considering the remote access to thermal system, “Sistema de Laboratorios a Distancia” (SLD) was used.

This paper describes the design and evaluation of a new platform created in order to improve the learning experience of bilateral control algorithms in teleoperation. This experimental platform, developed at Universidad Politécnica de Madrid, is used by the students of the Master on Automation and Robotics in the practices of the subject called “Telerobotics and Teleoperation”. The main objective is to easily implement different control architectures in the developed platform and evaluate them under different conditions to better understand the main advantages and drawbacks of each control scheme. So, the student's tasks are focused on adjusting the control parameters of the predefined controllers and designing new ones to analyze the changes in the behavior of the whole system. A description of the subject, main topics and the platform constructed are detailed in the paper. Furthermore, the methodology followed in the practices and the bilateral control algorithms are presented. Finally, the results obtained in the experiments with students are also shown.

This paper presents a new Haptic Rate-Position controller, which allows manipulating a slave robot in a large workspace. Haptic information is displayed to the user so as to be informed when a change in the operation mode occurs. This controller allows performing tasks in a large remote workspace by using a haptic device with a reduced workspace such as Phantom. Experimental results have been carried out using a virtual slave robot, simulated with Open Dynamics Engine (ODE). A real IFMIF (International Fusion Materials Irradiation Facility) remote handling task has been simulated. Its goal is to carry out remote manipulation of test irradiated materials in a nuclear environment. The proposed algorithm has been compared with the classic Position controller in a Pick & Place manipulation and has shown much better levels of effectiveness.

Nowadays, almost all kinesthetic haptic devices commercially available have been conceived and developed for just one interaction point. Many mechanical conflicts arise when trying to put to work together two or more conventional haptic devices. Due to collisions and other incompatibilities the resulting workspace when using these configurations is usually too small, making this a non-viable option. Therefore, a new multi-finger haptic device called MasterFinger 3 is developed to meet the demands of freedom of movement within a wide workspace. The aim of the new haptic device presented in this paper is to allow an intuitive human-machine interaction in a natural fashion. This interaction permits the end-user to control and operate an advanced robotic hand precisely. In order to reach this purpose, the attitude of at least three fingers should be measured. Moreover, the attitude of the other two fingers can be deduced from those measurements. Consequently, the mechanical design and the electronic architecture are described in detail in the following sections.

The complexity in the execution of cooperative tasks is high due to the fact that a robot team requires movement coordination at the beginning of the mission and continuous coordination during the execution of the task. A variety of techniques have been proposed to give a solution to this problem assuming standard mobile robots. This work focuses on presenting the execution of a cooperative task by a modular robot team. The complexity of the task execution increases due to the fact that each robot is composed of modules which have to be coordinated in a proper way to successfully work. A combined tight and loose cooperation strategy is presented and a bar-pushing example is used as a cooperative task to show the performance of this type of system.

The modeling of a modular robot system is complex due to its capability of changing and forming countless robot configurations with a set of modules. The greater the number of modules added to the structure, more complicated is the robot and its modeling. A methodology to represent modular robot configurations as a fixed shape robot is presented. The equivalent model is obtained through the characterization of the module, graph theory and geometric modeling of robots. After the transformation, the resulting kinematic and dynamic parameters are similar to the traditional model of a single configuration robot. Once the equivalent model is obtained, it can be used within different robot applications.

This paper describes a modular robot system design SMART, based on three types of modules for urban search tasks. The system attempts to give a quick solution to natural and man-made disaster emergencies. It allows for rapid and cost-effective design and fabrication. The approach is based on the use of an inventory of three types of modules i.e., power and control module, joint module, and specialized module. They are interchangeable in different ways to form different robot configurations for a variety of tasks. Forward and inverse kinematics from assembled robot configurations are analyzed. Description of control motion modes for human-modular robot system interaction is presented.

The purpose of this paper is to analyze in some depth the kinematic behaviour of the human hand, in order to obtain simplified human hand models with the minimum and optimal number of Degrees of Freedom (DoF), and thus achieving an efficient manipulation task. The statistical analysis is carried out using Principal Components Analysis (PCA). Power and precision grasps are obtained with the use of a Cyberglove and a human hand model with 24 DoF. Finally, these experiments are used to evaluate the best DoF for an appropriate manipulation.

This paper describes an automated system that inspects the external surface of an aircraft section. To test the system developed, a part similar to airplane section 19 has been used. The inspection system is based on a robot carrying a headset made up of ultrasound sensors. Data are processed by an ultrasound processing program which maps the mechanical features of the inspected surface. The main requirement of the inspection is to perform the proper contact between the ultrasound sensors and the inspected surface. In order to guarantee suitable contact between them, a robot trajectory correction system running in real time has been applied. It is based on the measures given by four optical sensors located on the sides of the ultrasound headset. The integration and synchronization of the devices mentioned above make it possible to obtain an adequate inspection system for the external aircraft fuselage. Furthermore, inspection time has been significantly reduced; at the same time a fully-automated system has been developed to inspect the whole aircraft section surface.

Human users can obtain information about the physical properties of an object through direct manipulation with one or two hands. Object manipulation of virtual objects using force feedback haptic interfaces is very challenging due to current technological constrains that often affect the information obtained by the user. Here, we describe the Master Finger 2 (MF2), a force feedback device which allows manipulation of one or more objects with one or two hands. We use experimental data to evaluate the performance of MF2 based on its capability to simulate effectively the weight of virtual objects. The results and implications for system design are discussed.

This paper describes a new methodology for designing bilateral controllers based on transparency that applies a modified scheme of control by state of convergence. The design is based on modelling the behavior of the master and slave which regard state space equations, and also taking into account that perfect transparency cannot be reached. This methodology allows designing the controllers in order to obtain the convergence between the state of the master and the slave. Furthermore, it consequently provides a higher degree of transparency to the operator. The paper explains criteria in achieving convergence between the master and the slave, and so as with transparency on a steady state. A set of equations that calculate controller gains have been obtained by applying such criteria. In order to verify this new methodology, a master-slave system of 3 DoF have been used.

A parallel robot testbed based on Stewart-Gough platform calledactive-Helideckis designed, developed and tested as a helicopter floating Helideck. The objective of this testbed is to show the advantages of helicopters that use an active Helideck upon landing on and taking off from ships or from offshore structures. Active-helideck compensates simulated movements of a ship at sea. The main goal of this study is to maintain the robotpsilas end effector (Helideck) in a quasi-static position in accordance to an absolute inertial frame. Compensation is carried out through the coordinate action of its six prismatic actuators in function of an inertial measurement unit. Moreover, the simulation of the sea movement is done by a parallel robot calledshipplatformwith three degrees of freedom. The ship platform is built with a vertical oscillation along the z axis, i.e. heave, and rotates on remaining axes, i.e. roll and pitch. Active Helideck is able to compensate simulated movements by considering the ship as an inertial frame as observed in the experiment.

This work is focused on obtaining efficient human hand models that are suitable for manipulation tasks. A 24 DoF kinematic model of the human hand is defined to realistic movements. This model is based on the human skeleton. Dynamic and Static constraints have been included in order to improve the movement realism. Two simplified hand models with 9 and 6 DoF have been developed according to the constraints predefined. These simplified models involve some errors in reconstructing the hand posture. These errors are calculated with respect to the 24 DoF model and evaluated according to the hand gestures. Finally, some criteria are defined to select the hand description best suited to the features of the manipulation task.

This paper shows the advantages of having a modular system with 3-DoF spherical actuator in the base module to perform tasks that require displacement and object manipulation. Having 3-DoF actuator improves the complexity of coordination patterns and control algorithms of the modular system more relevantly as compared to having only 1 or 2DoF actuator in the module. Nevertheless, modules with actuators of only 1 or 2DoF require more modules to be assembled together in order to achieve complex tasks. Experiments performed with RobMAT modular system proves that a 3DoF actuator in the module is better, because tasks such as displacement, obstacle climbing and object manipulation, can be efficiently carried out with systems of 2 modules, 4 modules and 6 (maximum) modules connected together.

This paper presents a discrete design and control method of teleoperation systems. The design method is based on the state space formulation and it allows to obtain the control gains for any teleoperation system where the master and the slave manipulators would be represented by nth-order discrete linear transfer functions. The control method allows that the slave follows the master through the state convergence between the master and the slave. The method is able also to establish the desire dynamics of this convergence and the dynamics of the slave manipulator. The advantage of this method is that it can be applied to control a teleoperation system where the master and the slave are modeled by transfer functions of different order. In order to validate the control method, simulation results are shown considering a teleoperation system where the master and the slave are modeled by discrete transfer functions of different order

This paper presents the operations of a 3-D camera and a video processing device aiming at the improved performance of teleoperated tasks. The video processing device has two operation modes: a) one for the stereoscopic image visualization, and b) other for the blending of video and graphic images. Main results are shown; considering specially the influence of the different video frequencies, corresponding to PAL and VGA formats. Finally, two telerobotic applications, on which these devices have been successfully applied, are described.

Outage-free maintenance of electrical power lines has been conducted throughout the years in order to fulfil the increasing demand of uninterrupted power supply. In these techniques care regarding electric shock and falls must be exercised while the work is performed on or near energized elements. In order to increase the safety and comfort of the workers as well as the overall efficiency, a teleoperated system for live-line maintenance, called ROBTET, has been developed for the electrical utility IBERDROLA. ROBTET is described in detail in the paper. It has two hydraulic-driven master-slave teleoperated manipulators on top of an insulated boom over a truck. The operator commands the manipulators from a cabin on the truck via a pair of master arms, while he receives visual feedback through a vision system. Multimedia display, voice commands, stereo vision and force-feedback are some of the new features implemented in order to achieve the required telepresence of the operator and hence increase the performance of the system.