Grasping and manipulation are two of the most important hand functions that allow people to efficiently execute activities of daily living. Over the last years, many robotic devices have been proposed to assist people who suffer from neurological conditions by enhancing their grasping capabilities. In this work, we focus on the development of a robotic exoskeleton glove that can increase the grasp stability and the force exertion capabilities of the user by employing soft, telescopic, inflatable structures on the palmar side of the hand. Also, the proposed design employs a camera and an object identification system to facilitate the development of a shared control scheme that simplifies the operation of the device. The experiments demonstrate that the soft robotic exoskeleton glove can successfully execute semi-autonomous grasps and that the soft telescopic structures can increase the total exerted grasping forces by more than 40% when inflated.

Flexible flat cables (FFC) are very popular for connecting different components in modern electronics (e.g., mobile phones, laptops, tablets, etc.). The manipulation of FFCs typically relies on highly trained workers that spend hours performing the same repetitive processes, or on autonomous robotic systems that are equipped with simple clamping mechanisms or pneumatically driven suction cups. Such robotic systems are difficult to program and reprogram and often rely on sophisticated sensing elements and complicated control laws. Moreover, the performance and robustness of such systems is far from sufficient, hindering their mass adoption. The manipulation of FFCs is also quite challenging. A good gripper should be able to pinch the cable steadily and execute insertion tasks of the cable connector with ease. The suction cup based solution is a good approach for holding the cable, but it makes the cable connector insertion very challenging as it can only apply limited shear forces. In this paper, we propose a locally-adaptive, pneumatic, parallel-jaw robot gripper equipped with fingertips that are able to both pinch the cable with a soft clamping mechanism and roll the cable surface on the soft fingertip structure until it reaches the desired connector. The gripper base accommodates a camera that allows for the recognition and pose estimation of the flat, flexible cables and other electronic components. The gripper is of low-cost and low-complexity and it can facilitate the efficient and robust execution of FFC grasping and assembly tasks.

Electromyography (EMG) is a wearable, noninvasive, commonly used method for measuring the human muscular activations from the surface of the skin. In this work, we present a pilot study that focuses on the formulation of a shared control framework to facilitate the simplified execution of Electromyography (EMG) based telemanipulation tasks with a robotic platform. The framework combines a Random Forests (RF) regression method with a compliance controller that relies on the force measurements collected with a force-torque sensor. The RF regression efficiently maps the myoelectric activations of the human muscles to corresponding human wrist positions. Then, a teleoperation process is used to control the robot arm end-effector’s position, utilizing the human wrist position estimations. The examined application involves semi-autonomous cleaning of a whiteboard surface with the proposed framework. The compliance controller guarantees that a desired contact force will always be maintained on the whiteboard surface during task execution. This ensures that any EMG based decoding inaccuracies will not drive the robot away from the cleaning plane. Essentially, the system projects the EMG based estimation on the cleaning plane. The shared control framework offers robust performance, with minimal training and calibration required.

Robot grasping and manipulation planning in unstructured and dynamic environments is heavily dependent on the attributes of manipulated objects. Although deep learning approaches have delivered exceptional performance in robot perception, human perception and reasoning are still superior in processing novel object classes. Moreover, training such models requires large datasets that are generally expensive to obtain. This work combines crowdsourcing and gamification to leverage human intelligence, enhancing the object recognition and attribute estimation aspects of robot perception. The framework employs an attribute matching system that encodes visual information into an online puzzle game, utilizing the collective intelligence of players to expand an initial attribute database and react to real-time perception conflicts. The framework is deployed and evaluated in a proof-of-concept application for enhancing object recognition in autonomous robot grasping and a model for estimating the response time is proposed. The obtained results demonstrate that given enough players, the framework can offer near real-time labeling of novel objects, based purely on visual information and human experience.

Many studies have focused on Virtual Reality (VR) frameworks for remotely controlling robotic systems. Although VR systems have been used to teleoperate robots in simple scenarios, their effectiveness in terms of accuracy, speed, and usability has not been rigorously evaluated for complex tasks that require accurate trajectories. In this work, an Enhanced Virtual Reality (EVR) framework for robotic teleoperation is evaluated to assess if it can be efficiently used in complex tasks that require accurate control of the robotic end-effector. The environment and the employed robot are captured using RGB-D cameras, while the remote user controls the motion of the robot with VR controllers. The captured data are transmitted and reconstructed in 3D so as to allow the remote user to monitor the task execution progress in real time, using a VR headset. The EVR system is compared with two other interface alternatives: i) teleoperation in pure VR (the model of the robot is rendered with respect to its real joint states), and ii) teleoperation in EVRR (the model of the robot is superimposed on the real robot). The results show that pure point cloud interfaces suffer from visualization issues, reducing the effectiveness of the robot teleoperation. However, the accuracy and user experience can be greatly improved by including the robot model.

The ongoing global healthcare crisis has amplified the need for automation of manual tasks in several industries and service sectors. Simple household tasks such as tidying and cleaning are in high demand, with only a few robotic platforms capable of performing them due to the mobility, workspace, and dexterity requirements. This work presents ARoA, an autonomous robotic assistant that can execute complex tasks in industrial, service, and home environments. It is equipped with two lightweight, compliant, 7 degree of freedom arms and a pair of adaptive end-effectors that enable efficient execution of a wide range of tasks. Due to the linear rail based torso system that supports the arms, the ARoA offers exceptional flexibility in terms of reachable workspace. A framework for vision-based execution of tidying and cleaning tasks is also proposed and integrated in the platform. The efficiency of the ARoA platform was experimentally validated through two everyday life applications: i) picking up and tidying randomly scattered household objects and ii) cleaning of common surfaces.

Small, lighter-than-air (LTA) robotic airship platforms offer an alternative to the more common, rotor-based Unmanned Aerial Vehicles (UAVs). LTA vehicles are attractive due to their inherent safety, mobility, low power consumption, and extended flight times, making them suitable for operation in populated indoor environments. This paper explores the use of shared control strategies for teleoperation of miniature indoor robotic airships, paired with an autonomous landing and charging system. The teleoperation scheme passes the operator inputs to the airship actuators in a standardized manner, allowing for simple integration with various control input devices. Specifically, this work employs three different devices with distinctive user input mechanics. The autonomous landing system relies on ArUco markers and an on-board camera for state estimation. The developed docking station relies on a magnet-based winch mechanism that catches and pulls the airship to the appropriate position for charging. Finally, the shared control teleoperation framework is validated through a series of experiments involving user-guided indoor exploration and autonomous landing, with promising results.

Current trends in industrial automation favor agile systems that allow adaptation to rapidly changing task requirements and facilitate customized production in smaller batches. This work presents a flexible manufacturing system relying on compliance control, CAD based localization, and a multi-modal gripper to enable fast and efficient task programming for assembly operations. CAD file processing is employed to extract component pose data from 3D assembly models, while the system's active compliance compensates for errors in calibration or positioning. To minimize retooling delays, a novel gripper design incorporating both a parallel jaw element and a rotating module is proposed. The developed system placed first in the manufacturing track of the Robotic Grasping and Manipulation Competition of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2019, experimentally validating its efficiency.

With the emergence of agile manufacturing in highly automated industrial environments, the demand for efficient robot adaptation to dynamic task requirements is increasing. For assembly tasks in particular, classic robot programming methods tend to be rather time intensive. Thus, effectively responding to rapid production changes requires faster and more intuitive robot teaching approaches. This work focuses on combining programming by demonstration with path optimization and local replanning methods to allow for fast and intuitive programming of assembly tasks that requires minimal user expertise. Two demonstration approaches have been developed and integrated in the framework, one that relies on human to robot motion mapping (teleoperation based approach) and a kinesthetic teaching method. The two approaches have been compared with the classic, pendant based teaching. The framework optimizes the demonstrated robot trajectories with respect to the detected obstacle space and the provided task specifications and goals. The framework has also been designed to employ a local replanning scheme that adjusts the optimized robot path based on online feedback from the camera-based perception system, ensuring collision-free navigation and the execution of critical assembly motions. The efficiency of the methods has been validated through a series of experiments involving the execution of assembly tasks. Extensive comparisons of the different demonstration methods have been performed and the approaches have been evaluated in terms of teaching time, ease of use, and path length.

In this work, a Mixed Reality (MR) system is evaluated to assess whether it can be efficiently used in teleoperation tasks that require an accurate control of the robot end-effector. The robot and its local environment are captured using multiple RGB-D cameras, and a remote user controls the robot arm motion through Virtual Reality (VR) controllers. The captured data is streamed through the network and reconstructed in 3D, allowing the remote user to monitor the state of execution in real time through a VR headset. We compared our method with two other interfaces: i) teleoperation in pure VR, with the robot model rendered with the real joint states, and ii) teleoperation in MR, with the rendered model of the robot superimposed on the actual point cloud data. Preliminary results indicate that the virtual robot visualization is better than the pure point cloud for accurate teleoperation of a robot arm.

The concept of teleoperation has been studied since the advent of robotics and has found use in a wide range of applications, including exploration of remote or dangerous environments (e.g., space missions, disaster management), telepresence based time optimisation (e.g., remote surgery) and robot learning. While a significant amount of research has been invested into the field, intricate manipulation tasks still remain challenging from the user perspective due to control complexity. In this paper, we propose an intuitive, affordances oriented telemanipulation framework for a dual robot arm hand system. An object recognition module is utilised to extract scene information and provide grasping and manipulation assistance to the user, simplifying the control of adaptive, multi-fingered hands through a commercial Virtual Reality (VR) interface. The system's performance was experimentally validated in a remote operation setting, where the user successfully performed a set of bimanual manipulation tasks.

Teleoperation and telemanipulation have since the early years of robotics found use in a wide range of applications, including exploration, maintenance, and response in remote or hazardous environments, healthcare, and education settings. As the capabilities of robot manipulators grow, so does the control complexity and the remote execution of intricate manipulation tasks still remains challenging for the user. This paper proposes an intuitive telemanipulation framework based on electromyography (EMG) and fiducial marker based tracking that can be used with a dexterous robot arm hand system. The EMG subsystem captures the myoelectric activations of the user during the execution of specific hand postures and gestures and translates them into the desired grasp type for the robot hand. The pose of the tracked fiducial marker is used as a task-space goal for the robot end-effector. The system performance is experimentally validated in a remote operation setting, where the system successfully performs a telemanipulation task.