Categorical Distributional Reinforcement Learning (CDRL) has demonstrated superior sample efficiency in learning complex tasks compared to conventional Reinforcement Learning (RL) approaches. However, the practical application of CDRL is encumbered by challenging projection steps, detailed parameter tuning, and domain knowledge. This paper addresses these challenges by introducing a pioneering Continuous Distributional Model-Free RL algorithm tailored for continuous action spaces. The proposed algorithm simplifies the implementation of distributional RL, adopting an actor-critic architecture wherein the critic outputs a continuous probability distribution. Additionally, we propose an ensemble of multiple critics fused through a Kalman fusion mechanism to mitigate overestimation bias. Through a series of experiments, we validate that our proposed method provides a sample-efficient solution for executing complex continuous-control tasks.

Humanlike robotic hands, such as prosthetic hands, become more advanced as technology develops, giving us more lightweight, sophisticated solutions with multiple degrees of freedom. Alongside the hardware improvements, control systems and human machine interfaces are also important areas of research to ensure that the operation of robotic hands is intuitive and easy to master. In particular, amputees are frequently disappointed with the difficulty in controlling their prostheses, which can lead to prostheses rejection. One method that has been explored to reduce the effort and cognitive load on the user is to implement semi-autonomy via appropriate control schemes. In this paper, a semi-autonomous control framework is proposed employing lightmyography based decoding of grasping motions. The proposed framework makes use of video and IMU data so as to reduce the number of possible grasps (grasp affordances) based on the object detected and the hand orientation. The efficiency of the proposed framework has been experimentally validated in comparison to a manual control framework. Using the semi-autonomous framework, misclassifications decreased, leading to 17/20 successful reach to grasps motions executed compared to 7/20 for the manual control case for a single subject. The automatically positioned thumb functionality has also robustified grasping, allowing certain objects to be more dexterously interacted with.

The human hand is an extraordinary example of evolution, capable of performing a wide range of tasks from intricate manipulation tasks to powerful grasps. Replicating such versatility and dexterity in both robotic and prosthetic hands is a longstanding engineering challenge. Despite paramount efforts from academia and industry, robotic and prosthetic hands still fall behind their human counterparts in many aspects, including dexterity. Closing this gap is essential for robotic and prosthetic hand applications such as general humanoids, prostheses, service robotics, and human-robot interaction, in which the need for human-like capabilities is of high significance. This paper presents the design of a highly actuated, 24-DoF, tendon-driven anthropomorphic robotic hand testbed for dexterous manipulation tasks. The hand is designed to be lightweight, affordable, and accessible with readily available components. A series of experiments is conducted to evaluate the hand's design and performance. Results show that the hand possesses a comparable workspace to the Shadow Hand and high repeatability in finger movements, essential for accurate sim2real transfer.

Amputations and limb loss can have detrimental effects on personal well-being. Although prosthetic devices can offer significant benefits helping amputees regain some of the lost dexterity, they often lack the required affordability and durability. Current affordable prosthetic designs have trended towards underactuation, which contributes to stable grasping but is often characterized by low durability. In this paper, a new chain-driven, adaptive, underactuated finger design has been proposed for the development of affordable and highly durable prosthetic hands. The transmission mechanism used is composed of a steel roller chain and several routing sprockets. The finger phalanges are constructed of 3D printed PLA, and finger flexion is produced by pulling the internally routed roller chain. In total, six 3D printed PLA sprockets are used for chain routing, with a design emphasis on high force transmission. The performance of the proposed chaindriven finger was experimentally validated and compared with an analogous tendon-driven version. The metrics employed for this comparison were longevity, pinch grasp efficiency, force response, and maximum force capability. The chain-driven finger was shown to have a higher maximum transmissible force, better long term durability, and no issues related to elongation (such as tendon elongation). The cost to manufacture the chain-driven robotic finger is only 91 USD, making it an excellent solution for affordable prostheses.

Soft robotics leverages highly elastic, deformable materials to enable sensitive human-centered applications in medicine, rehabilitation, and assistance, as well as industrial applications like robotic grasping and load-bearing. One particular type of soft robotic actuator - the pneumatically-actuated, soft robotic, telescopic structure (PASTS) - is a relatively new concept that utilises compliant material and geometry reminiscent of traditional telescopes to produce linear motion and force exertion. Previous works on telescopic soft actuators have focused on specific applications rather than fundamental mechanics, creating a clear knowledge gap in understanding their design, dynamics, and dependencies. This paper provides an in-depth study of the fundamental design parameters of soft telescopic actuators, and examines the impacts of certain critical dimensions and geometries on the physical behaviour and capabilities of these soft actuators. It revolves around the exploration of the influence of telescopic structure’s length, wall thickness and number of rings on its inflation and deflation behaviour, lifting and lowering speed, and motion smoothness. By experimentally verifying these relationships using several design variations, it demonstrates that telescopic structures are capable of repeatable linear extension within 0.85 mm of precision. It also determines the varying degrees±by which each critical parameter affects the desirable properties of the actuator, allowing for telescopic structures to be tailored and optimised for specific applications.

Monitoring of waterways such as remote and hazardous rivers and streams is important so as to assess the impact of external factors including construction runoff or climate change. Versatile, autonomous robotic boats can offer excellent environmental inspection and monitoring solutions for remote, dangerous, or access protected water bodies but they have several shortcomings in terms of maneuverability. This paper proposes an environmental inspection system consisting of an autonomous data collection buoy which is designed to be deployed to inaccessible river systems using a drone. The system can perform a drop off and pickup of the buoy depending on the requirements of a particular location and monitoring task. Utilising the natural flow of the river the buoy autonomously steers down, using GPS and magnetometers so as to maintain the desired trajectory. The buoy is capable of measuring water temperature but it can also be equipped with a range of sensors such as water oxygen meter, sonar for river bed inspection, or turbidity for water clarity. This paper describes the system design, presents an analysis of the self-righting capabilities of the buoy, and shows a full system demonstration at the Ōrewa River in Auckland, New Zealand.

While autonomous racing performance in Time-Trial scenarios has seen significant progress and development, autonomous wheel-to-wheel racing and overtaking are still severely limited. These limitations are particularly apparent in real-life driving scenarios where state-of-the-art algorithms struggle to safely or reliably complete overtaking manoeuvres. This is important, as reliable navigation around other vehicles is vital for safe autonomous wheel-to-wheel racing. The F1Tenth Competition provides a useful opportunity for developing wheel-to-wheel racing algorithms on a standardised physical platform. The competition format makes it possible to evaluate overtaking and wheel-to-wheel racing algorithms against the state-of-the-art. This research presents a novel racing and overtaking agent capable of learning to reliably navigate a track and overtake opponents in both simulation and reality. The agent was deployed on an F1Tenth vehicle and competed against opponents running varying competitive algorithms in the real world. The results demonstrate that the agent’s training against opponents enables deliberate overtaking behaviours with an overtaking rate of 87% compared 56% for an agent trained just to race.

Solar power is becoming an increasingly popular option for energy production in commercial and private applications. While installing solar panels (photovoltaic cells) in a stationary configuration is simple and inexpensive, such a setup fails to maximise their potential solar energy production. Single- and dual-axis sun trackers automatically adjust the tilt angle of photovoltaic cells so as to directly face towards sun, but these also come with their own drawbacks such as increased cost and weight. This paper presents SOL, a soft-robotic, dual-axis, sun-tracking mechanism for improved solar panel efficiency. The proposed design was built to be compact, portable, and lightweight, and it utilises closed-loop control for the intelligent actuation of a set of soft telescopic structures that raise and tilt the solar panels in the direction of the sun. The performance of the proposed solar tracking platform was experimentally validated in terms of its maximum elevation at different azimuths and its ability to balance different loads. The result is a device that provides solar panel users with an accessible, affordable, and convenient means of increasing the efficiency of their solar energy system.

This work introduces the New Dexterity modular, dexterous, anthropomorphic, open-source, bimanual manipulation platform (OpenBMP) that is designed for research and rapid experimentation in robot grasping, dexterous manipulation, and bimanual manipulation. The platform combines adaptive and hybrid actuation systems with lockable joints, facilitating transitions between the execution of delicate and forceful tasks. Antagonistic tendon-driven elbows and inline actuator transmissions reduce the system’s inertial mass while enhancing energy efficiency and overall performance. Leveraging 3D printing and carbon fiber reinforced manufacturing of core parts, the platform is easy to replicate and highly modular. This paper presents the details of the design, the actuation principles, and the experimental validation of the efficiency of the platform with the execution of complex teleoperation and telemanipulation tasks. The designs, the electronics, and the code are open-sourced to allow replication by others.

Inspection of powerlines is a hard problem that requires humans to operate in remote locations and dangerous conditions. This paper proposes a quadcopter unmanned aerial vehicle (UAV) equipped with rolling-capable perching mechanisms and a depth-vision system for the purpose of autonomous power line inspection. The perching mechanism grips onto the power line, allowing the UAV to withstand external forces such as wind disturbances. Once engaged and applying the desired gripping force, the perching mechanism requires no power through the use of a ratcheting serial elastic transmission, allowing the UAV to perch indefinitely. The depth-vision system automates the perching and unperching procedures by estimating the position and pose of the UAV relative to the powerline. These measurements are sent to a local position controller that guides the UAV to and from the power line. Once perched, rollers in the fingers of the perching mechanism drive the UAV along the powerline, providing a close-up platform for inspection equipment. The proposed system was tested in an outdoor testing environment and shown to autonomously perch and unperch from a steel cable. The grippers force application was analysed and the UAVs powerless robust perch is demonstrated by total disconnect of power while perched. These results suggest that such a system could be a valuable tool for the upkeep of electricity networks.

Versatile, autonomous robotic boats can offer excellent environmental inspection and monitoring solutions for remote, dangerous, hard to reach, or access protected water bodies. This paper introduces such a platform in the form of an autonomous, cost-effective, waterjet-powered robotic trimaran. Motivated by the need for an efficient aquatic monitoring, particularly in Aotearoa - New Zealand’s diverse environments, the trimaran provides an efficient, low-cost, and easy to replicate alternative to resource-intensive research vessels. The proposed platform, costs $600-1,500 USD to develop (depending on the sensing system configuration), weighs under 5 kg, and excels in bathymetry and water quality testing. The trimaran can reach speeds of up to 2 m/s offering obstacle avoidance of natural features, such as rocks. Utilizing off-the-shelf components and 3D printing technology, the proposed platform offers excellent reproducibility and robustness while operating in shallow waters with its jet propulsion system. The paper presents in detail the design characteristics, the sensing system employed, testing results focusing on bathymetry, and highlights the ability of vessel and the potential for future research and data collection.

Reinforcement Learning (RL) has been widely used to solve tasks where the environment consistently provides a dense reward value. However, in real-world scenarios, rewards can often be poorly defined or sparse. Auxiliary signals are indispensable for discovering efficient exploration strategies and aiding the learning process. In this work, inspired by intrinsic motivation theory, we postulate that the intrinsic stimuli of novelty and surprise can assist in improving exploration in complex, sparsely rewarded environments. We introduce a novel sample-efficient method able to learn directly from pixels, an image-based extension of TD3 with an autoencoder called NaSA-TD3. The experiments demonstrate that NaSA-TD3 is easy to train and an efficient method for tackling complex continuous-control robotic tasks, both in simulated environments and real-world settings. NaSA-TD3 outperforms existing state-of-the-art RL image-based methods in terms of final performance without requiring pre-trained models or human demonstrations.

The advancement in robotic grasping and manipulation has elicited an increased research interest in the development of household robots capable of performing a plethora of complex tasks. These advancements require the shift of robotics research from a laboratory setting to dynamic and unstructured home environments. In this work, we focus on a comprehensive data collection and analysis of key attributes involved in the selection of grasping and manipulation strategies for the successful execution of kitchen tasks. An unprecedented dataset that comprises over 7 hours of high-definition videos that were analyzed to classify more than 10,000 kitchen activities annotated with 24 attributes each has been created. Machine learning techniques were employed to automate the annotation process partially by extracting grasp types, hand, and object information from the videos. The annotated dataset was analyzed using clustering algorithms to identify underlying patterns. This study also identifies key attributes and specific data that require focus during data collection based on inter-subject variability. The insights from this study can be used to improve the speed, quality, and effectiveness of data collection. It also helps identify the strategies employed by the humans for the execution of kitchen tasks and transfer the necessary skills to a robotic end-effector enabling it to complete the tasks autonomously or collaborate with humans.

Robots have predominantly been used in automating tasks in structured industrial environments, however, with the advances in technology they are starting to take part in roles in dynamic everyday life scenarios. As a result, the tasks executed by robotic systems will also grow in sophistication. Grasping and dexterous manipulation are critical aspects that allow humans to execute these sophisticated tasks, enabling them to interact with their environment. As such, emulating the human hand can be advantageous for interacting with a world designed for humans. However, directly replicating the anatomical structure of the hand produces designs that are fully actuated, expensive, and which require sophisticated controls and sensing to operate efficiently. In this paper, we present two different versions of the New Dexterity adaptive, humanlike robot hand that is capable of executing robust caging grasps under a wide range of environmental uncertainties (e.g., object pose uncertainties). One of the versions has a classic, fixed thumb base while the second one incorporates an additional degree of freedom at the thumb base, which enables a translational motion for repositioning the thumb and adjusting the aperture. This design choice enhances the inhand manipulation capabilities of the robot hand, improving also the power grasping capabilities for larger objects. The performances of the proposed robot hand designs are experimentally validated and compared through three different tests: i) grasping experiments involving everyday-life objects, ii) force experiments that evaluate their force exertion capabilities, and iii) in-hand manipulation experiments that demonstrate and compare their dexterity.

Model Free Reinforcement Learning (MFRL) has shown significant promise for learning dexterous robotic manipulation tasks, at least in simulation. However, the high number of samples, as well as the long training times, prevent MFRL from scaling to complex real-world tasks. Model- Based Reinforcement Learning (MBRL) emerges as a potential solution that, in theory, can improve the data efficiency of MFRL approaches. This could drastically reduce the training time of MFRL, and increase the application of RL for real- world robotic tasks. This article presents a study on the feasibility of using the state-of-the-art MBRL to improve the training time for two real-world dexterous manipulation tasks. The evaluation is conducted on a real low-cost robot gripper where the predictive model and the control policy are learned from scratch. The results indicate that MBRL is capable of learning accurate models of the world, but does not show clear improvements in learning the control policy in the real world as prior literature suggests should be expected.

Over the last decade, a plethora of soft robotic devices have been proposed for the execution of complex grasping and dexterous manipulation tasks. Tasks requiring such increased dexterity are typically executed using fully-actuated, rigid end-effectors equipped with sophisticated sensing and controlled with complex control laws. The new class of soft robotic devices offers an alternative to the traditional end-effectors and facilitates the development of robotic grasping and manipulation solutions that are lightweight, safe to interact with, affordable, and easy to use and control. Within the class of soft robotic grippers and hands, promising recent developments were made in ultra-affordable, even disposable mechanisms based on origami and kirigami structures. This paper proposes a new kirigami-inspired robotic gripper geometry employing compression-based actuation. The compression actuation fundamentally differentiates this new design class from previous kirigami grippers, resulting in more compact robotic grippers with superior grasping capabilities. In particular, we investigate how the shapes of the internal cuts of the kirigami geometries can affect the gripper performance in terms of force exertion and grasping capabilities. A series of experiments are conducted to understand better the working principles behind this new type of kirigami grippers and experimentally validate their efficacy in the execution of complex, everyday life tasks. Further demonstrations of the gripper's capabilities include the pick-and-placing of human hair, egg yolk, and even liquids.

Over the last decades, significant research effort has been put into creating Electromyography (EMG) based controllers for intuitive, hands-free control of robotic arms and hands. To achieve this, machine learning models have been employed to decode human motion and intention using EMG signals as input and to deliver several applications, such as prosthesis control using gesture classification. Despite the advances introduced by new deep learning techniques, real-time control of robot arms and hands using EMG signals as input still lacks accuracy, especially when a plethora of gestures are included as labels in the case of classification. This has been observed to be due to the noise and non-stationarity of the EMG signals and the increased dimensionality of the problem. In this paper, we propose an intuitive, affordances-oriented EMG-based telemanipulation framework for a robot arm-hand system that allows for dexterous control of the device. An external camera is utilized to perform scene understanding and object detection and recognition, providing grasping and manipulation assistance to the user and simplifying control. Object-specific Transformer-based classifiers are employed based on the affordances of the object of interest, reducing the number of possible gesture outputs, dividing and conquering the problem, and resulting in a more robust and accurate gesture decoding system when compared to a single generic classification model. The performance of the proposed system is experimentally validated in a remote telemanipulation setting, where the user successfully performs a set of dexterous manipulation tasks.

Telemanipulation is widely used in robotics applications, ranging from maintenance of various industrial systems to search and rescue response in remote and/or hazardous environments. Human operators are often responsible for the control of such robotic systems. However, these remote interactions require highly trained and experienced operators owing to their complex nature. Semi-autonomous systems are presented as an alternative to complex and counter-intuitive manual systems, combining decoded user intentions with autonomous control modules. This paper proposes a semi-autonomous framework for robotic telemanipulation that employs Electromyography (EMG) based motion decoding and potential fields to execute complex object stacking tasks with a dexterous robot arm-hand system. Even though simple EMG-based teleoperation is promising, the signals are often noisy leading to induced randomness and control errors. To assist the user during task executions, potential fields are utilized to avoid obstacles and guide the robot end-effector toward the objects of interest, thus reducing the cognitive load on the user and the need for accurate predictions. The user's motion is decoded from the myoelectric activations of the human upper arm and upper torso using a Random Forest-based regression methodology. The objects are detected in the environment with an external camera that provides their goal poses to the potential fields scheme. EMG control and potential fields work in a synergistic manner simplifying the system's operation. The framework performance is experimentally validated in real-time experiments involving complex cube and cylinder stacking tasks.

Soft robotic devices have been popular in handling intricate grasping and dexterous manipulation tasks, serving as an alternative to conventional, rigid end-effectors. These devices are relatively simple, lightweight, and cost-effective. Recently, kirigami based structures have been used to create low-cost and disposable soft robotic grippers and hands. These grippers undergo a complex post-contact reconfiguration and conform to an object's shape and size during grasping. In this paper, we explore this new class of soft robotic grippers by utilising them for single-grasp object classification and grasping force estimation. We install simplistic sensors on both the gripper and the actuation system to estimate the state of the kirigami gripper, and the collected data features are employed to train Random Forest models for identifying the grasped object. The classifier trained exhibits a high accuracy of 98 % in discriminating objects of various shapes. When handling food items, the classifier achieves an accuracy of 94 %, while in classifying transparent objects, the classifier obtained again a high accuracy of 97 %. Finally, object-specific force estimation models are triggered based on the classification decision of the Random Forest model to estimate the grasping force exerted by the gripper. These positive outcomes demonstrate the kirigami based robotic gripper's potential for object classification in a variety of circumstances, particularly where vision systems are not available or not reliable.

Unmanned aerial vehicles (UAVs) have revolutionised various industries, such as agriculture, remote sensing, and infrastructure inspection. To explore new designs and improve UAV flight performance, roboticists are seeking inspiration from nature. In this paper, we present a bioinspired tailsitter UAV utilizing shape-morphing wings with aerofoil-shaped artificial feathers. The design of the UAV is inspired by the shape and motion of bird wings, which can change their shape and span to adapt to different flight conditions. The pigeon's wing skeletal structure serves as the basis for the design, and the wing was developed to be fully tendon-driven employing a single motor for each side. The wings can contract and extend, resulting in a contraction ratio of 49% of the extended wing span. In hovering flight mode, the wing contraction shows a 42% decrease in drag for improved wind disturbance rejection. Wind tunnel testing characterises the wing's aerodynamic performance, revealing significant deflection at high angles of attack due to the articulated skeletal structure. The wings demonstrate low power consumption, averaging only 5.1 W during morphing in experiments. Finally, we demonstrate the wing's robustness through outdoor flight experiments. The research findings provide insights into the potential of bioinspired designs for tailsitter UAVs and offer a promising avenue for future research in this field.

The advent of collaborative industrial and house-hold robotics has blurred the demarcation between the human and robot workspace. The capability of robots to function efficiently alongside humans requires new research to be conducted in dynamic environments as opposed to the traditional well-structured laboratory. In this work, we propose an efficient skill transfer methodology comprising intuitive interfaces, efficient optical tracking systems, and compliant control of robotic arm-hand systems. The lightweight wearable interfaces mounted with robotic grippers and hands allow the execution of dexterous activities in dynamic environments without restricting human dexterity. The fiducial and reflective markers mounted on the interfaces facilitate the extraction of positional and rotational information allowing efficient trajectory tracking. As the tasks are performed using the mounted grippers and hands, gripper state information can be directly transferred. The hardware-agnostic nature and efficiency of the proposed interfaces and skill transfer methodology are demonstrated through the execution of complex tasks that require increased dexterity, writing and drawing.

Robot grasping and manipulation allow robots to interact with their environments and execute a plethora of complex tasks that require increased dexterity (e.g., open a door, push buttons, collect and transpose objects, etc.). Collecting data of such activities is of paramount importance as it allows roboticists to create new methods and models that will facilitate the execution of sophisticated tasks. In this paper, we propose new wearable, lightweight, low-cost human machine interfaces that improve the efficiency of the data collection process for both robotic grasping and manipulation by offering intuitive and simplified control of the employed robotic grippers and hands. In particular, two different types of interfaces are proposed: i) a handle-based forearm stabilized interface that uses a waist-linkage system to provide weight support for bulky and heavy robotic end-effectors and ii) a palm-mounted interface that can accommodate smaller and lightweight grippers and hands, offering more agility in the control and positioning of these devices. Both interfaces are equipped with appropriate sliders, joysticks, and buttons that facilitate the control of the multiple degrees of freedom of the employed end-effectors and appropriate cameras that allow for object detection, identification, and object pose estimation.

Grasping and manipulation are two of the most important hand functions that allow people to efficiently execute activities of daily living. Over the last years, many robotic devices have been proposed to assist people who suffer from neurological conditions by enhancing their grasping capabilities. In this work, we focus on the development of a robotic exoskeleton glove that can increase the grasp stability and the force exertion capabilities of the user by employing soft, telescopic, inflatable structures on the palmar side of the hand. Also, the proposed design employs a camera and an object identification system to facilitate the development of a shared control scheme that simplifies the operation of the device. The experiments demonstrate that the soft robotic exoskeleton glove can successfully execute semi-autonomous grasps and that the soft telescopic structures can increase the total exerted grasping forces by more than 40% when inflated.

A popular solution for connecting different components in modern electronics, such as mobile phones, laptops, tablets, etc, is the use of flexible flat cables (FFC). Typically, it takes hours of repetition from a highly trained worker, or a high precision autonomous robot with specialised end effectors to reliably manage the installation of these cables. Human workers are prone to error, and cannot work endlessly without a break, while the robots often come with a significant expense, and require a substantial amount of time to program and reprogram. Additionally, the use of sophisticated sensing elements further increases the complexity of the required control system. As a result, the performance and robustness of such systems is far from sufficient, hindering their mass adoption. The manipulation of FFCs is also quite challenging. In this work, we focus on the robotic manipulation of a plethora of flexible cables, proposing a multi-modal gripper with locally-dexterous tips and active fingernails. The fingers of the gripper are equipped with: i) locally-dexterous fingertips that accommodate manipulation-capable degrees of freedom, ii) a combination of Nitinol-based active fingernails and suction cups that allow picking up and handling of cables that rest on flat surfaces, and iii) compliant finger-pads that conform to the object surface to increase grasping stability. The proposed robotic gripper is equipped with a camera and a perception system that allow for the execution of complex cable manipulation and assembly tasks in dynamic environments.

Grasping and manipulation are complex and demanding tasks, especially when executed in dynamic and unstructured environments. Typically, such tasks are executed by rigid articulated end-effectors, with a plethora of actuators that need sophisticated sensing and complex control laws to execute them efficiently. Soft robotics offers an alternative that allows for simplified execution of these demanding tasks, enabling the creation of robust, efficient, lightweight, and affordable solutions that are easy to control and operate. In this work, we introduce a new class of soft, kirigami-based robotic grippers, we study their post-contact behavior, and we investigate different cut patterns for their development. We follow an experimental approach in which several designs are proposed and employed in a series of grasping and force exertion tests to compare their capabilities and post-contact behavior. The results of such experiments indicate a clear relationship between degree of reconfiguration and grasping force, and provide key insights into the effect of the cut patterns in the performance of the designs. These findings are then used in the design process of an improved version of multi-layer, disposable kirigami grippers that are fabricated employing simple 3D printed layers and silicone rubber using the concept of Hybrid Deposition Manufacturing (HDM). A series of experimental results demonstrate that the proposed design and manufacturing methods can enable the creation of soft, kirigami-based grippers with superior grasping capabilities that can handle delicate, contaminated, and everyday life objects and can even be disposed off in an automated way (e.g., after handling hazardous materials, such as medical waste).

To communicate, the ~ 1.5 million Americans living with deafblindess use tactile American Sign Language (t-ASL). To provide Deafßilind (DB) individuals with a means of using their primary communication language without the use of an interpreter, we developed an assistive technology that promotes their autonomy. The TATUM (Tactile ASL Translational User Mechanism) anthropomorphic arm hand system leverages previous developments of a fingerspelling hand to sign more complex ASL words and phrases. The TATUM hand-wrist system is attached onto a 4 DOF robot arm and a human motion recognition and human to robot gesture transfer framework is used for signing recognition and replication. In particular, signing trajectories based on vision-based motion capture data from a sign demonstrator were used to control the robot's actuators. The performance of the system was evaluated through tactile based sign recognition performed by a blinded user and for its accuracy with novice, sighted users.

Soft robots are capable of effortlessly adapting to their environment using elastic materials that impart structural compliance into their designs, allowing them to execute complex tasks with minimal sensing and control. However, soft robots cannot exert high forces and can only handle low deformation forces. These characteristics typically limit their applicabil-ity to tasks that require delicate interactions. In this work, we present a mechanically programmable, variable stiffness, jamming actuator based on an articulated mesh structure. The proposed actuator can elastically bend when it is not activated but compresses to attain a pre-programmed shape that is determined by the mesh geometry of the multi-layer jamming architecture when pressure is applied to the silicone pouch containing it. Unlike traditional jamming structures the utilisation of the articulated mesh structure facilitates elastic deformations past the yield point when jammed. The actuator can become >27 times stiffer than its relaxed configuration when exposed to only 90 kPa pressure. We demonstrate the efficiency of this actuator by developing variable stiffness joints that can be used to create: i) underactuated, tendon driven robotic grippers and soft, disposable robotic grippers that exhibit increased dexterity and ii) wearable, affordable, lightweight elbow exoskeleton systems that can assist humans in holding heavy objects with minimal effort.

For the development of muscle-machine interfaces (MuMIs), researchers have relied mainly on Electromyography (EMG) signals. However, these signals require complex hardware systems, as well as specialized signal processing and feature extraction methods. To overcome these issues, in our previous work, we proposed a novel MuMI for decoding human intention and motion, called Lightmyography (LMG). To improve the performance of this interface even further, in this work, we employ two novel deep learning techniques called Temporal Multi-Channel Transformer (TMC-T) and Temporal Multi-Channel Vision Transformer (TMC-ViT) for the classification of hand gestures based on the LMG data. The performance of these two Transformer-based methods is evaluated and compared with other well-known deep learning and classical machine learning methods. This work also addresses the influence of varying parameters defined during the training phase of decoding models, such as the size and shape of the input data packet. A series of data augmentation techniques were also employed to generate synthetic data and increase the dataset size so as to train deep learning models more efficiently.

This work presents an adaptive prosthetic training gripper with a compact, variable stiffness differential mechanism and a vision-based shared control scheme that relies on a Lightmyography (LMG) interface to trigger the selected grasps. The gripper incorporates three monolithic adaptive fingers manufactured using the concept of Hybrid Deposition Manufacturing (HDM) and includes a gear drive system that allows two of the finger bases to rotate, implementing abduction / adduction and thereby increasing the available grasping workspace. The fingers are actuated through a compact, series-elastic differential mechanism that reduces the total number of required actuators to only two. The developed adaptive robotic gripper is operated using a vision-based myoelectric control framework that utilizes an RGB camera and a Convolutional Neural Network (CNN) for object detection and classification as well as for grasp selection and an LMG muscle machine interface for grasp triggering. The efficiency of the proposed gripper and the control framework have been experimentally validated through a series of complex grasping experiments executed using a plethora of eveyday life objects.

Flexible flat cables (FFC) are very popular for connecting different components in modern electronics (e.g., mobile phones, laptops, tablets, etc.). The manipulation of FFCs typically relies on highly trained workers that spend hours performing the same repetitive processes, or on autonomous robotic systems that are equipped with simple clamping mechanisms or pneumatically driven suction cups. Such robotic systems are difficult to program and reprogram and often rely on sophisticated sensing elements and complicated control laws. Moreover, the performance and robustness of such systems is far from sufficient, hindering their mass adoption. The manipulation of FFCs is also quite challenging. A good gripper should be able to pinch the cable steadily and execute insertion tasks of the cable connector with ease. The suction cup based solution is a good approach for holding the cable, but it makes the cable connector insertion very challenging as it can only apply limited shear forces. In this paper, we propose a locally-adaptive, pneumatic, parallel-jaw robot gripper equipped with fingertips that are able to both pinch the cable with a soft clamping mechanism and roll the cable surface on the soft fingertip structure until it reaches the desired connector. The gripper base accommodates a camera that allows for the recognition and pose estimation of the flat, flexible cables and other electronic components. The gripper is of low-cost and low-complexity and it can facilitate the efficient and robust execution of FFC grasping and assembly tasks.

Electromyography (EMG) is a wearable, noninvasive, commonly used method for measuring the human muscular activations from the surface of the skin. In this work, we present a pilot study that focuses on the formulation of a shared control framework to facilitate the simplified execution of Electromyography (EMG) based telemanipulation tasks with a robotic platform. The framework combines a Random Forests (RF) regression method with a compliance controller that relies on the force measurements collected with a force-torque sensor. The RF regression efficiently maps the myoelectric activations of the human muscles to corresponding human wrist positions. Then, a teleoperation process is used to control the robot arm end-effector’s position, utilizing the human wrist position estimations. The examined application involves semi-autonomous cleaning of a whiteboard surface with the proposed framework. The compliance controller guarantees that a desired contact force will always be maintained on the whiteboard surface during task execution. This ensures that any EMG based decoding inaccuracies will not drive the robot away from the cleaning plane. Essentially, the system projects the EMG based estimation on the cleaning plane. The shared control framework offers robust performance, with minimal training and calibration required.

Robot grasping and manipulation planning in unstructured and dynamic environments is heavily dependent on the attributes of manipulated objects. Although deep learning approaches have delivered exceptional performance in robot perception, human perception and reasoning are still superior in processing novel object classes. Moreover, training such models requires large datasets that are generally expensive to obtain. This work combines crowdsourcing and gamification to leverage human intelligence, enhancing the object recognition and attribute estimation aspects of robot perception. The framework employs an attribute matching system that encodes visual information into an online puzzle game, utilizing the collective intelligence of players to expand an initial attribute database and react to real-time perception conflicts. The framework is deployed and evaluated in a proof-of-concept application for enhancing object recognition in autonomous robot grasping and a model for estimating the response time is proposed. The obtained results demonstrate that given enough players, the framework can offer near real-time labeling of novel objects, based purely on visual information and human experience.

Lecturers of Engineering courses around the world are struggling to increase the engagement of students through the introduction of appropriate hands-on activities and assignments. In Biomechatronics and Robotics courses these assignments typically focus on how certain devices are designed, modelled, fabricated, or controlled. The hardware for these assignments is usually purchased by some external vendor and the students only get the chance to analyze it or program it, so as to execute a useful task (e.g., programming mobile robots to perform path following tasks). Student engagement can be increased by instructing the students to prepare the hardware for their assignment. This also increases the sense of ownership of the project outcomes. In this paper, we present how a robotic gripper / hand design project and the introduction of a grasping and manipulation competition as a course assignment, can significantly increase the student engagement and their understanding of the taught concepts. The presented best practices have been trialed over the last four years in two different courses (one undergraduate and one postgraduate) of the Department of Mechanical Engineering at the University of Auckland in New Zealand. For the particular assignment the students were asked to fully develop a robotic gripper or hand from scratch using a single actuator (only the actuator and the power electronics were provided). The performance of the developed devices was assessed through the participation in a grasping and manipulation competition. All the details of the proposed assignment are presented, hoping that they could help other lecturers and teachers to prepare similar activities.

Many studies have focused on Virtual Reality (VR) frameworks for remotely controlling robotic systems. Although VR systems have been used to teleoperate robots in simple scenarios, their effectiveness in terms of accuracy, speed, and usability has not been rigorously evaluated for complex tasks that require accurate trajectories. In this work, an Enhanced Virtual Reality (EVR) framework for robotic teleoperation is evaluated to assess if it can be efficiently used in complex tasks that require accurate control of the robotic end-effector. The environment and the employed robot are captured using RGB-D cameras, while the remote user controls the motion of the robot with VR controllers. The captured data are transmitted and reconstructed in 3D so as to allow the remote user to monitor the task execution progress in real time, using a VR headset. The EVR system is compared with two other interface alternatives: i) teleoperation in pure VR (the model of the robot is rendered with respect to its real joint states), and ii) teleoperation in EVRR (the model of the robot is superimposed on the real robot). The results show that pure point cloud interfaces suffer from visualization issues, reducing the effectiveness of the robot teleoperation. However, the accuracy and user experience can be greatly improved by including the robot model.

Micro Aerial Vehicles (MAV) with Vertical Takeoff and Landing (VTOL) capabilities, such as quadrotors, have offered significant value to many research fields and markets. However, only recently, MAV began to be explored as systems capable of interacting with the environment, performing manipulation tasks, and participating in more versatility-demanding operations. Pursuing the goal of turning flying machines into more versatile instruments, many researchers have resorted to using tilting rotor mechanisms to create new aerial vehicle concepts. Nevertheless, most such new concepts are bulky and lack the required versatility, and are restricted to particular applications. In this work, we address these issues by proposing a novel coaxial, versatile, modular tilt-rotor UAV concept. The Omnirotor platform can apply its full thrust in any direction, regardless of the frame’s orientation where it is mounted. The platform does not have any limitations regarding rotation’s range. It can change its thrust direction continuously without needing to unwind back to a specific configuration. With the addition of control surfaces between the coaxial rotors, the Omnirotor is turned into a functional VTOL MAV with hovering capabilities that can be used as a ground vehicle, a UAV, and an all-terrain vehicle.

With an increasing number of robotic and prosthetic devices, there is a need for intuitive interfaces which enable the user to efficiently interact with them. The conventional interfaces are generally bulky and unsuitable for dynamic and unstructured environments. An alternative to the traditional interfaces is the class of Muscle-Machine Interfaces (MuMIs) that allow the user to have an embodied interaction with the devices they are controlling. In this work, we present a wearable, lightweight, Forcemyography (FMG) based armband for Human-Machine Interaction fabricated entirely out of 3D-printed parts and silicone components. The armband uses six force sensing units, each housing an Force Sensitive Resistor (FSR) sensor. The capabilities of the armband are evaluated while decoding four different gestures (pinch, power, tripod, extension) and rest state and its performance is compared with a state-of-the-art Electromyography (EMG) bioamplifier. The decoding performance of the decoding models trained on the data acquired from the armband is significantly better than the performance of the models trained on raw EMG data. The hardware design and the related processing software, are disseminated in an open-source manner.

Design optimization can lead to the development of robotic end-effectors with optimal grasping and dexterous, in-hand manipulation capabilities. In particular, the finger link dimensions have been identified as one of the primary design parameters that affects the performance of a robotic gripper. The ability of a gripper to manipulate objects is mainly attributed to the interaction between a set of coordinated fingers. This coordination is primarily affected by the inter-finger distance. This paper presents a framework for finding an appropriate distance between the finger bases of a two-fingered robotic gripper so as to increase the dexterous manipulation workspace for a range of object sizes. To do that, a parallel multi-start search algorithm is employed to solve a multiparametric optimization problem. The results demonstrate that different distances lead to completely different workspace shapes and that the ratio defined by the area of the optimized workspace (nominator) and the union of all workspaces (denominator) is always significantly less than 1. This means that the area of the union of all workspaces is always larger than the area of the "optimized" workspace. Based on these results a multi-modal robotic gripper with movable finger bases was developed. The proposed gripper can vary the distance between the finger bases online and it offers an increased dexterous manipulation workspace without sacrificing grasping performance.

Robot grasping and dexterous, in-hand manipulation allow robots to interact with their surroundings and execute a plethora of complex tasks such as pushing buttons, opening doors, and interacting with electrical appliances. In robotics, such complicated tasks are typically executed by multi-fingered end-effectors that are heavy, rigid, and expensive, employing numerous degrees of freedom and actuation. In this paper, we focus on the analysis, design, and development of a multi-grasp, reconfigurable, five fingered, anthropomorphic robot hand that can facilitate the execution of both robust grasping and dexterous manipulation tasks in service robotics and industrial automation applications. The robot hand is composed of eight actuators driving eighteen degrees of freedom with a telescoping mechanism and opposable thumb and pinky fingers to produce multiple anthropomorphic and non-anthropomorphic configurations for grasping and manipulation tasks. The reconfigurable finger base frames allow the hand to transform and utilize its degrees of actuation in an optimal manner to overcome its underactuated limitations. The underactuated robot hand is designed with a human hand structure that takes advantage of objects specifically designed for human operation (e.g., tool or handles with ergonomics for the human hand). This allows the system to better operate within a human-centered environment. The effectiveness of the proposed device is experimentally validated through three different tests: i) grasping experiments involving everyday-life objects, ii) force experiments that assess the force exertion capabilities of the hand in different finger base frame configurations, and iii) demonstration of in-hand object manipulation capabilities. The proposed hand weighs 1.28 kg and has a cost of approximately $1920 USD. The device is capable of exerting up to 14.3 N of contact force during pinch grasping and a maximum of 150.6 N power grasping.

Over the last decade, adaptive tendon driven devices have gained an increased interest from the research community for their lightweight, compact, and affordable design features attributed to the utilisation of underactuation, differential mechanisms, and structural compliance. Although adaptive tendon driven devices are capable of efficiently executing stable grasps under significant object pose uncertainties with simplistic control algorithms, they lack the controllability over individual fingers in comparison to traditional fully actuated designs. In this paper, we focus on the development of a selectively lockable differential mechanism that is powered through a small and low torque servo to provide increased autonomy to highly underactuated and adaptive prosthetic hands, without compromising the weight, cost, and compactness of the device. The proposed prosthetic hand is experimentally validated through four tests: i) grasping posture and gesture execution experiments, ii) grasping experiments with everyday life objects, iii) force exertion experiments, and iv) Electromyography (EMG) based control of the prosthetic hand.

The ongoing global healthcare crisis has amplified the need for automation of manual tasks in several industries and service sectors. Simple household tasks such as tidying and cleaning are in high demand, with only a few robotic platforms capable of performing them due to the mobility, workspace, and dexterity requirements. This work presents ARoA, an autonomous robotic assistant that can execute complex tasks in industrial, service, and home environments. It is equipped with two lightweight, compliant, 7 degree of freedom arms and a pair of adaptive end-effectors that enable efficient execution of a wide range of tasks. Due to the linear rail based torso system that supports the arms, the ARoA offers exceptional flexibility in terms of reachable workspace. A framework for vision-based execution of tidying and cleaning tasks is also proposed and integrated in the platform. The efficiency of the ARoA platform was experimentally validated through two everyday life applications: i) picking up and tidying randomly scattered household objects and ii) cleaning of common surfaces.

Differential mechanisms allow the designers of robotic and prosthetic grippers and hands to create devices that require a minimal number of motors in order to grasp a plethora of everyday life objects, leading to light-weight, compact, and low-cost implementations. The working principle of differential mechanisms is simple. They allow the distribution of the forces exerted by a single actuator to multiple outputs (e.g., fingers). This reduction in the number of motors leads to underactuation, which is the use of fewer motors than the available degrees of freedom. But differentials need also to be power-efficient, compact, adaptive, and lightweight. Most of the existing solutions lack at least one of these attributes. In this paper, we focus on the design, modeling, and development of a compact, adaptive, series elastic differential. The proposed mechanism consists of four elastic elements connected in series with the four output attachments. The compression of the elastic elements during grasping allows the gripper or hand to conform to the object’s shape. The efficiency of the differential mechanism is experimentally validated using two different types of experiments, measuring: i) the maximum achievable tension load at the outputs, and ii) the maximum achievable compliance of a single output when all other outputs are blocked. The proposed differential has been employed for the development of a gripper and its efficiency has been assessed by executing grasping tasks with several everyday life objects. The device can be easily replicated using additive manufacturing and off-the-shelf materials and is disseminated in an open-source manner.

Small, lighter-than-air (LTA) robotic airship platforms offer an alternative to the more common, rotor-based Unmanned Aerial Vehicles (UAVs). LTA vehicles are attractive due to their inherent safety, mobility, low power consumption, and extended flight times, making them suitable for operation in populated indoor environments. This paper explores the use of shared control strategies for teleoperation of miniature indoor robotic airships, paired with an autonomous landing and charging system. The teleoperation scheme passes the operator inputs to the airship actuators in a standardized manner, allowing for simple integration with various control input devices. Specifically, this work employs three different devices with distinctive user input mechanics. The autonomous landing system relies on ArUco markers and an on-board camera for state estimation. The developed docking station relies on a magnet-based winch mechanism that catches and pulls the airship to the appropriate position for charging. Finally, the shared control teleoperation framework is validated through a series of experiments involving user-guided indoor exploration and autonomous landing, with promising results.

Electromyography (EMG) based interfaces have been used in various robotics studies ranging from teleoperation and telemanipulation applications to the EMG based control of prosthetic, assistive, or robotic rehabilitation devices. But most of these studies have focused on the decoding of user's motion or on the control of the robotic devices in the execution of simple tasks (e.g., grasping tasks). In this work, we present a learning scheme that employs High Density Electromyography (HD-EMG) sensors to decode a set of dexterous, in-hand manipulation motions (in the object space) based on the myoelectric activations of human forearm and hand muscles. To do that, the subjects were asked to perform roll, pitch, and yaw motions manipulating two different cubes. The first cube was designed to have a center of mass coinciding with the geometric center of the cube, while for the second cube the center of mass was shifted 14 mm to the right (off-centered design). Regarding the acquisition of the myoelectric data, custom HD-EMG electrode arrays were designed and fabricated. Using these arrays, a total of 89 EMG signals were extracted. The object motion decoding was formulated as a regression problem using the Random Forests (RF) technique and the muscle importances were studied using the inherent feature variables importance calculation procedure of the RF. The muscle importance results show that different subjects use different strategies to execute the same motions on same object when the weight is off-centered. Finally, the decoded motions were used to control a five fingered robotic hand in a proof-of-concept application.

Over the last years, hand exoskeletons have become a popular and efficient technical solution for assisting people that suffer from neurological and musculoskeletal diseases and enhance the capabilities of healthy individuals. These devices can vary from rigid and complex structures to soft, lightweight, wearable gloves. Despite the significant progress in the field, most existing solutions do not provide the same dexterity as the healthy human hand. In this paper, we focus on the development of a hybrid (tendon-driven and pneumatic), lightweight, affordable, wearable exoskeleton glove equipped with abduction/adduction capabilities and a pneumatic telescopic extra thumb that increases grasp stability. The efficiency of the proposed device is experimentally validated through three different types of experiments: i) abduction/adduction tests, ii) force exertion experiments that capture the maximum forces that can be applied by the proposed device, and iii) grasp quality assessment experiments that focus on the effect of the inflatable thumb on enhancing grasp stability. The hybrid assistive glove considerably improves the grasping capabilities of the user, being able to exert the forces required to assist people in the execution of activities of daily living.

Although soft robots are a good alternative to rigid, traditional robots due to their intrinsic compliance and environmental adaptability, there are several drawbacks that limit their impact, such as low force exertion capability and low resistance to deformation. For this reason, soft structures of variable stiffness have become a popular solution in the field to combine the benefits of both soft and rigid designs. In this paper, we develop laminar jamming flexure joints that facilitate the development of adaptive robot grippers with variable stiffness. Initially, we propose a mathematical model of the laminar jamming structures. Then, the model is experimentally validated through bending tests using different materials, pressures, and number of layers. Finally, the soft, laminar jamming structured are employed to develop variable stiffness flexure joints for two different adaptive robot grippers. Bending profile analysis and grasping tests have demonstrated the benefits of the proposed jamming structures and the capabilities of the designed grippers.

Current trends in industrial automation favor agile systems that allow adaptation to rapidly changing task requirements and facilitate customized production in smaller batches. This work presents a flexible manufacturing system relying on compliance control, CAD based localization, and a multi-modal gripper to enable fast and efficient task programming for assembly operations. CAD file processing is employed to extract component pose data from 3D assembly models, while the system's active compliance compensates for errors in calibration or positioning. To minimize retooling delays, a novel gripper design incorporating both a parallel jaw element and a rotating module is proposed. The developed system placed first in the manufacturing track of the Robotic Grasping and Manipulation Competition of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2019, experimentally validating its efficiency.

Robots and intelligent industrial systems that focus on sorting or inspection of products require end-effectors that can grasp and manipulate the objects surrounding them. The capability of such systems largely depends on their ability to efficiently identify the objects and estimate the forces exerted on them. This paper presents an underactuated, compliant, and lightweight hyper-adaptive robot gripper that can efficiently discriminate between different everyday life objects and estimate the contact forces exerted on them during a single grasp, using vision-based techniques. The hyper-adaptive mechanism consists of an array of movable steel rods that get reconfigured conforming to the geometry of the grasped object. The proposed object identification and force estimation techniques are model-free and do not rely on time consuming object exploration. A series of experiments have been carried out to discriminate between 12 different everyday life objects and estimate the forces exerted on a dynamometer. During each grasp, a series of images are captured that detect the reconfiguration of the hyper-adaptive grasping mechanism. These images are then used by an image processing algorithm to extract the required information about the gripper reconfiguration, classify the object grasped using a Random Forests (RF) classifier, and estimate the amount of force being exerted. The employed RF classifier gives a prediction accuracy of 100%, while the results of the force estimation techniques (Neural Networks, Random Forests, and 3rd order polynomial) range from 94.7% to 99.1%.

To facilitate the development of a new generation of Virtual Reality systems and their introduction in everyday life applications, new intuitive, immersive methods of interfacing have to be developed. Over the years, Electromyography (EMG) based interfaces have been utilized for unobtrusive interaction with computer systems. However, previous EMG studies have not explored the continuous decoding of the effects of human motion (e.g., manipulated object behavior) in simulated and virtual environments. In this work, we present an EMG based learning framework that can allow for an immersive interaction with Virtual Reality environments. To do that, EMG activations from the muscles of the forearm and the hand were acquired during the execution of object manipulation tasks in a virtual world along with the motion of the object. The virtual world was visualized using an HTC Vive VR headset, while the hand motions were tracked with a dataglove equipped with magnetic motion capture sensors. The object motion decoding was formulated as a regression problem using the Random Forests methodology. The study shows that the object motion can be successfully decoded using the EMG activations, despite the lack of haptic feedback.

With the emergence of agile manufacturing in highly automated industrial environments, the demand for efficient robot adaptation to dynamic task requirements is increasing. For assembly tasks in particular, classic robot programming methods tend to be rather time intensive. Thus, effectively responding to rapid production changes requires faster and more intuitive robot teaching approaches. This work focuses on combining programming by demonstration with path optimization and local replanning methods to allow for fast and intuitive programming of assembly tasks that requires minimal user expertise. Two demonstration approaches have been developed and integrated in the framework, one that relies on human to robot motion mapping (teleoperation based approach) and a kinesthetic teaching method. The two approaches have been compared with the classic, pendant based teaching. The framework optimizes the demonstrated robot trajectories with respect to the detected obstacle space and the provided task specifications and goals. The framework has also been designed to employ a local replanning scheme that adjusts the optimized robot path based on online feedback from the camera-based perception system, ensuring collision-free navigation and the execution of critical assembly motions. The efficiency of the methods has been validated through a series of experiments involving the execution of assembly tasks. Extensive comparisons of the different demonstration methods have been performed and the approaches have been evaluated in terms of teaching time, ease of use, and path length.

In this work, a Mixed Reality (MR) system is evaluated to assess whether it can be efficiently used in teleoperation tasks that require an accurate control of the robot end-effector. The robot and its local environment are captured using multiple RGB-D cameras, and a remote user controls the robot arm motion through Virtual Reality (VR) controllers. The captured data is streamed through the network and reconstructed in 3D, allowing the remote user to monitor the state of execution in real time through a VR headset. We compared our method with two other interfaces: i) teleoperation in pure VR, with the robot model rendered with the real joint states, and ii) teleoperation in MR, with the rendered model of the robot superimposed on the actual point cloud data. Preliminary results indicate that the virtual robot visualization is better than the pure point cloud for accurate teleoperation of a robot arm.

Autonomous intelligent systems outperform human workers in an expanding range of domains, typically those in which success is a function of speed, precision and repeatability. However, many cognitive tasks remain beyond the reach of automation. In this work, we propose the use of video games to crowdsource the cognitive versatility and creativity of human players to solve complex problems in industrial automation and robotics applications. To do so, we introduce a theoretical framework in which robotics problems are embedded into video game environments and gameplay from crowds of players is aggregated to inform robot actions. Such a framework could enable a future of synergistic human-machine collaboration for industrial automation, in which members of the public not only freely offer the fruits of their intelligent reasoning for productive use, but have fun whilst doing so. There is also potential for significant negative consequences surrounding safety, accountability and ethics if great care is not taken in the implementation. Further work is needed to explore these wider implications, as well as to develop the technical theory behind the framework and build prototype applications.

Adaptive, underactuated and compliant robot hands have received an increased interest over the last decade. Possible applications of these systems range from the development of simple grippers for industrial automation to the creation of anthropomorphic devices that can be used as prosthetic hands. These hands are particularly capable of extracting stable grasps even under significant object pose or other environmental uncertainties, due to the underactuation and the structural compliance of their designs. Despite the increased interest and the promising performance, adaptive hands suffer from several disadvantages and drawbacks. For example, the use of underactuation can lead to a post-contact reconfiguration of the fingers that compromises the force exertion capabilities of the system during pinch grasping. In this paper, we focus on the design, modelling, development, and evaluation of an adaptive robot gripper that uses magnets to adjust the reconfiguration profile of the fingers. The effect of the magnets increases the gripper's force exertion capabilities in pinch grasps, without compromising the full/caging grasps. The efficiency of the proposed gripper is experimentally validated through two different tests: i) a contact force test that compares the results of a theoretical model with the actual experimental results and ii) a grasping test that assesses the force exertion capabilities and the reconfiguration behaviour of the adaptive fingers for different implementations of the magnetic joints.

The concept of teleoperation has been studied since the advent of robotics and has found use in a wide range of applications, including exploration of remote or dangerous environments (e.g., space missions, disaster management), telepresence based time optimisation (e.g., remote surgery) and robot learning. While a significant amount of research has been invested into the field, intricate manipulation tasks still remain challenging from the user perspective due to control complexity. In this paper, we propose an intuitive, affordances oriented telemanipulation framework for a dual robot arm hand system. An object recognition module is utilised to extract scene information and provide grasping and manipulation assistance to the user, simplifying the control of adaptive, multi-fingered hands through a commercial Virtual Reality (VR) interface. The system's performance was experimentally validated in a remote operation setting, where the user successfully performed a set of bimanual manipulation tasks.

Current grasping methods for aerial vehicles are slow, inaccurate and they cannot adapt to any target object. Thus, they do not allow for on-the-fly, ultra-fast grasping. In this paper, we present a passive closing, adaptive robot hand design that offers ultra-fast, aerial grasping for a wide range of everyday objects. We investigate alternative uses of structural compliance for the development of simple, adaptive robot grippers and hands and we propose an appropriate quick release mechanism that facilitates an instantaneous grasping execution. The quick release mechanism is triggered by a simple distance sensor. The proposed hand utilizes only two actuators to control multiple degrees of freedom over three fingers and it retains the superior grasping capabilities of adaptive grasping mechanisms, even under significant object pose or other environmental uncertainties. The hand achieves a grasping time of 96 ms, a maximum grasping force of 56 N and it is able to secure objects of various shapes at high speeds. The proposed hand can serve as the end-effector of grasping capable Unmanned Aerial Vehicle (UAV) platforms and it can offer perching capabilities, facilitating autonomous docking.

Adaptive robot hands are typically created by introducing structural compliance either in their joints (e.g., implementation of flexure joints) or in their finger-pads. In this paper, we present a series of alternative uses of structural compliance for the development of simple, adaptive, compliant and/or under-actuated robot grippers and hands that can efficiently and robustly execute a variety of grasping and dexterous, in-hand manipulation tasks. The proposed designs utilize only one actuator per finger to control multiple degrees of freedom and they retain the superior grasping capabilities of the adaptive grasping mechanisms even under significant object pose or other environmental uncertainties. More specifically, in this work, we introduce, discuss, and evaluate: a) the concept of compliance adjustable motions that can be predetermined by tuning the in-series compliance of the tendon routing system and by appropriately selecting the imposed tendon loads, b) a design paradigm of pre-shaped, compliant robot fingers that adapt / conform to the object geometry and, c) a hyper-adaptive finger-pad design that maximizes the area of the contact patches between the hand and the object, maximizing also grasp stability. The proposed hands use mechanical adaptability to facilitate and simplify the efficient execution of robust grasping and dexterous, in-hand manipulation tasks by design.

Teleoperation and telemanipulation have since the early years of robotics found use in a wide range of applications, including exploration, maintenance, and response in remote or hazardous environments, healthcare, and education settings. As the capabilities of robot manipulators grow, so does the control complexity and the remote execution of intricate manipulation tasks still remains challenging for the user. This paper proposes an intuitive telemanipulation framework based on electromyography (EMG) and fiducial marker based tracking that can be used with a dexterous robot arm hand system. The EMG subsystem captures the myoelectric activations of the user during the execution of specific hand postures and gestures and translates them into the desired grasp type for the robot hand. The pose of the tracked fiducial marker is used as a task-space goal for the robot end-effector. The system performance is experimentally validated in a remote operation setting, where the system successfully performs a telemanipulation task.

In this paper, we focus on the formulation of a hybrid methodology that combines analytical models, constrained optimization schemes, and machine learning techniques to simplify the execution of dexterous, in-hand manipulation tasks with adaptive robot hands. More precisely, the constrained optimization scheme is used to describe the kinematics of adaptive hands during the grasping and manipulation processes, unsupervised learning (clustering) is used to group together similar manipulation strategies, dimensionality reduction is used to either extract a set of representative motion primitives (for the identified groups of manipulation strategies) or to solve the manipulation problem in a low-d space and finally an automated experimental setup is used for unsupervised, automated collection of large data sets. We also assess the capabilities of the derived manipulation models and primitives for both model and everyday life objects, and we analyze the resulting manipulation ranges of motion (e.g., object perturbations achieved during the dexterous, in-hand manipulation). We show that the proposed methods facilitate the execution of fingertip-based, within-hand manipulation tasks while requiring minimal sensory information and control effort, and we demonstrate this experimentally on a range of adaptive hands. Finally, we introduce DexRep, an online repository for dexterous manipulation models that facilitate the execution of complex tasks with adaptive robot hands.

Robots need to use their end-effectors not only to grasp and manipulate objects but also to understand the environment surrounding them. Object identification is of paramount importance in robotics applications, as it facilitates autonomous object handling, sorting, and quality inspection. In this paper, we present a new hyper-adaptive robot hand that is capable of discriminating between different everyday objects, as well as `model' objects with the same external geometry but varying material, density, or volume, with a single grasp. This work leverages all the benefits of simple, adaptive grasping mechanisms (robustness, simplicity, low weight, adaptability), a Random Forests classifier, tactile modules based on barometric sensors, and radar technology offered by the Google Soli sensor. Unlike prior work, the method does not rely on object exploration, object release or re-grasping and works for a wide variety of everyday objects. The feature space used consists of the Google Soli readings, the motor positions and the contact forces measured at different time instances of the grasping process. The whole approach is model-free and the hand is controlled in an open-loop fashion, achieving stable grasps with minimal complexity. The efficiency of the designs, sensors, and methods has been experimentally validated with experimental paradigms involving model and everyday objects.

In this paper, we present a methodology based on constrained optimization methods for estimating and compensating for post-contact parasitic object motions for underactuated, compliant robot hands and for deriving stable, minimal effort grasps to try to minimize these movements. To do so, we compute the object motions for different hand designs, object shapes, and object sizes and we synthesize appropriate robot arm trajectories that eliminate them, even in hands with complex flexure-based compliant members. The effectiveness of the proposed methods is validated using a seven DOF robot arm (Barrett WAM) and a range of compliant underactuated robot hands (Yale OpenHand models T42PP, T42PF, and T42FF).

A new class of simple, adaptive, under-actuated and compliant robot hands has recently attracted the interest of the robotics community. The under-actuated mechanisms and the structural compliance used in these hands facilitate and robustify not only grasping but also the execution of dexterous, in-hand manipulation tasks. Another significant characteristic of the particular hands is that they are able to efficiently grasp a wide range of everyday life objects even under significant object pose uncertainties. However, these hands, are difficult to model due to kinematic constraints introduced by the underactuation and the use of complex flexure joints. Moreover, adaptive hands tend to reconfigure upon contact with the object surface, imposing certain parasitic object motions. In this paper, we propose a learning scheme that uses the contact force measurements collected from tactile sensors to estimate the post-contact reconfiguration of the hand-object system and the imposed parasitic object motion. The learning scheme's estimates are compared with “ground truth” data that describe the actual motion of the object and that are collected using a vision based motion capture system. The proposed learning scheme can be used with any type of adaptive robot hand and its efficiency is experimentally validated using extensive paradigms involving different hand designs and various everyday life objects.

Adaptive robot hands have changed the way we approach and think of robot grasping and manipulation. Traditionally, pinch, fingertip grasping and dexterous, in-hand manipulation tasks were executed with fully actuated, rigid robot hands and relied on analytic methods, computation of the hand object Jacobians and extensive numerical simulations for deriving optimal and minimal effort grasps. However, even insignificant uncertainties in the modeling space could render the extraction of candidate grasps or manipulation paths infeasible. Adaptive hands use underactuated mechanisms and structural compliance, facilitating by design the successful extraction of stable grasps and the robust execution of manipulation tasks, even under significant object pose or other environmental uncertainties. In this paper, we propose a methodology for the automated extraction of dexterous, in-hand manipulation strategies / primitives for adaptive hands. To do so, we use a constrained optimization scheme that describes the kinematics of adaptive hands during the grasping and manipulation processes, an automated experimental setup for data collection, a clustering technique that groups together similar manipulation strategies, and a dimensionality reduction technique that projects the robot kinematics to lower dimensional manifolds. In these manifolds, control is simplified and hand operation becomes more intuitive. In this work, we also assess the effect of the extracted manipulation primitives on the object pose perturbations. The efficiency of the proposed methods is experimentally verified for various adaptive robot hands. The extracted primitives can simplify the operation and control of the open-source robot hand designs of the Yale Open Hand project in dexterous manipulation tasks.

In this paper, we propose a hybrid methodology based on a combination of analytical, numerical and machine learning methods for performing dexterous, in-hand manipulation with simple, adaptive robot hands. A constrained optimization scheme utilizes analytical models that describe the kinematics of adaptive hands and classic conventions for modelling quasistatically the manipulation problem, providing intuition about the problem mechanics. A machine learning (ML) scheme is used in order to split the problem space, deriving task-specific models that account for difficult to model, dynamic phenomena (e.g., slipping). In this respect, the ML scheme: 1) employs the simulation module in order to explore the feasible manipulation paths for a specific hand-object system, 2) feeds the feasible paths to an experimental setup that collects manipulation data in an automated fashion, 3) uses clustering techniques in order to group together similar manipulation trajectories, 4) trains a set of task-specific manipulation models and 5) uses classification techniques in order to trigger a task-specific model based on the user provided task specifications. The efficacy of the proposed methodology is experimentally validated using various adaptive robot hands in 2D and 3D in-hand manipulation tasks.

The past decade has seen great progress in the development of adaptive, low-complexity, underactuated robot hands. An advantage of these hands is that they use under-constrained mechanisms and compliance, which facilitate grasping even under significant object pose uncertainties. However, for many minimal contact grasps such as precision fingertip grasps, these hands tend to move the object after a grasp is secured, to an equilibrium configuration determined by the elasticity of the mechanism and the contact forces exerted through the robot fingertips. In this paper, we present a methodology based on constrained optimization methods for deriving stable, minimal effort grasps for underactuated robot hands and compensating for post-contact, in-hand parasitic object motions. To do so, we compute the imposed object motions for different object shapes and sizes and we synthesize appropriate robot arm trajectories that eliminate them. The approach allows for the computation of these grasps and motions even for hands with complex, flexure-based, compliant members. The effectiveness of the proposed methods is validated using a redundant robot arm (Barrett WAM) and a two fingered, compliant, underactuated robot hand (Yale Open Hand model T42), for a series of simulated and experimental paradigms.

In this paper we present an open-source design for the development of low-complexity, anthropomorphic, underactuated robot hands with a selectively lockable differential mechanism. The differential mechanism used is a variation of the whiffletree (or seesaw) mechanism, which introduces a set of locking buttons that can block the motion of each finger. The proposed design is unique since with a single motor and the proposed differential mechanism the user is able to control each finger independently and switch between different grasping postures in an intuitive manner. Anthropomorphism of robot structure and motion is achieved by employing in the design process an index of anthropomorphism. The proposed robot hands can be easily fabricated using low-cost, off-the-shelf materials and rapid prototyping techniques. The efficacy of the proposed design is validated through different experimental paradigms involving grasping of everyday life objects and execution of daily life activities. The proposed hands can be used as affordable prostheses, helping amputees regain their lost dexterity.

In this paper we present a methodology for discriminating between different objects using only a single force closure grasp with an underactuated robot hand equipped with force sensors. The technique leverages the benefits of simple, adaptive robot grippers (which can grasp successfully without prior knowledge of the hand or the object model), with an advanced machine learning technique (Random Forests). Unlike prior work in literature, the proposed methodology does not require object exploration, release or re-grasping and works for arbitrary object positions and orientations within the reach of a grasp. A two-fingered compliant, underactuated robot hand is controlled in an open-loop fashion to grasp objects with various shapes, sizes and stiffness. The Random Forests classification technique is used in order to discriminate between different object classes. The feature space used consists only of the actuator positions and the force sensor measurements at two specific time instances of the grasping process. A feature variables importance calculation procedure facilitates the identification of the most crucial features, concluding to the minimum number of sensors required. The efficiency of the proposed method is validated with two experimental paradigms involving two sets of fabricated model objects with different shapes, sizes and stiffness and a set of everyday life objects.

In this paper we introduce an index for the quantification of anthropomorphism of robot arms. The index is defined as a weighted sum of specific metrics which evaluate the similarities between the human and robot arm workspaces, providing a normalized score between 0 (non-anthropomorphic artifacts) and 1 (human-identical artifacts). The human arm workspaces were extracted using data reported in anthropometry studies. The formulation is general enough to allow utilization in various applications, by adjusting the weighting factors according to the specifications of each study. The proposed methodology can be used for assessing the human-likeness of existing robot arms as well as to provide specifications for the design of new anthropomorphic robots and prosthetic devices. To assess the efficiency of the proposed methods a comparative analysis between five kinematically different robot arm models is conducted and simulated paradigms are presented.

The majority of the works on grasping consider both object as well as robot hand parameters to be accurately known and do not take into account the constraints imposed by the robot hand. In this paper, a complete methodology is proposed that handles the grasping problem under a wide range of uncertainties. Initially, we search for an acceptable posture that provides robustness against positioning inaccuracies and maximizes the ability of the robot hand to exert forces on the object. Subsequently, in order to secure the grasp stability, we also deal with the determination of sufficient contact forces. Finally, an appropriate tactile sensor setup, mounted on the robot hand, allow us to reduce the magnitude of uncertainty regarding the grasping parameters. The efficiency of our approach is validated through extensive experimental paradigms using a 15 DoF DLR/HIT II robotic hand attached at the end effector of a 7 DoF Mitsubishi PA10 robotic manipulator.

In this paper, we propose an optimization scheme for deriving task-specific force closure grasps for underactuated robot hands. Motivated by recent neuroscientific studies on the human grasping behavior, a novel grasp strategy is built upon past analysis regarding the task-specificity of human grasps, that also complies with the recent soft synergy model of underactuated hands. Our scheme determines an efficient force closure grasp (i.e., configuration and contact points/forces) with a posture compatible with the desired task, taking into consideration the mechanical and geometric limitations imposed by the design of the hand and the object shape. The efficiency of the algorithm is verified through simulated paradigms on a hypothetical underactuated hand with the kinematic model of the DLR/HIT II five fingered robot hand.

In this paper, we propose a robust model free control scheme of minimal complexity (it is a static scheme involving very few and simple calculations to output the control signal) for robotic manipulators, capable of achieving prescribed transient and steady state performance. No information regarding the robot dynamic model is employed in the design procedure. Moreover, the tracking performance of the developed scheme (i.e., convergence rate and steady state error) is a priori and explicitly imposed by a designer-specified performance function, and is fully decoupled by both the control gains selection and the robot dynamic model. In that respect, the selection of the control gains is only confined to adopting those values that lead to reasonable control effort. Finally, two experimental studies in the joint and the Cartesian workspace clarify the design procedure and verify its performance and robustness against external disturbances.

In this paper, we propose a complete methodology for deriving task-specific force closure grasps for multifingered robot hands under a wide range of uncertainties. Given a finite set of external disturbances representing the task to be executed, the concept of Q distance is introduced in a novel way to determine an efficient grasp with a task compatible hand posture (i.e., configuration and contact points). Our approach takes, also, into consideration the mechanical and geometric limitations imposed by the robotic hand design and the object to be grasped. In addition, incorporating our recent results on grasping [1], the ability of the robot hand to exert the required contact forces is maximized and robustness against positioning inaccuracies and object uncertainties is established. Finally, the efficiency of our approach is verified through an experimental study on the 15 DoF DLR/HIT II robotic hand attached at the end effector of the 7 DoF Mitsubishi PA10 robotic manipulator.

In this paper, we propose a novel image based visual servoing scheme that imposes prescribed transient and steady state response on the image feature coordinate errors and satisfies the visibility constraints that inherently arise owing to the limited field of view (FOV) of cameras. Visualizing the aforementioned performance specifications as error bounds, the key idea is to provide an error transformation that converts the original constrained problem into an equivalent unconstrained one, the stabilization of which proves sufficient to achieve prescribed performance guarantees and satisfy the inherent visibility constraints. The performance of the developed scheme is a priori and explicitly imposed by certain designer-specified performance functions, and is fully decoupled by the control gains selection, thus simplifying the control design. Moreover, its computational complexity proves significantly low. It is actually a static scheme involving very few and simple calculations to output the control signal, which enables easily its implementation on fast embedded control platforms. Finally, real-time experiments using an eye-in-hand robotic system verify the theoretical findings.

In this paper we present a series of design directions for the development of affordable, modular, light-weight, intrinsically-compliant, underactuated robot hands, that can be easily reproduced using off-the-shelf materials. The proposed robot hands, efficiently grasp a series of everyday life objects and are considered to be general purpose, as they can be used for various applications. The efficiency of the proposed robot hands has been experimentally validated through a series of experimental paradigms, involving: grasping of multiple everyday life objects with different geometries, myoelectric (EMG) control of the robot hands in grasping tasks, preliminary results on a grasping capable quadrotor and autonomous grasp planning under object position and shape uncertainties.

In this paper a comparative analysis between the human and three robotic hands is conducted. A series of metrics are introduced to quantify anthropomorphism and assess robot's ability to mimic the human hand. In order to quantify anthropomorphism we choose to compare human and robot hands in two different levels: comparing finger phalanges workspaces and comparing workspaces of the fingers base frames. The final score of anthropomorphism uses a set of weighting factors that can be adjusted according to the specifications of each study, providing always a normalized score between 0 (non-anthropomorphic) and 1 (human-identical). The proposed methodology can be used in order to grade the human-likeness of existing and new robotic hands, as well as to provide specifications for the design of the next generation of anthropomorphic hands. Those hands can be used for human robot interaction applications, humanoids or even prostheses.

In this paper teleoperation and telemanipulation with a robot arm (Mitsubishi PA-10) and a robot hand (DLR/HIT 2) is performed, using a human to robot motion mapping scheme that guarantees anthropomorphism. Two position trackers are used to capture position and orientation of human end-effector (wrist) and human elbow in 3D space and a dataglove to capture human hand kinematics. Then the inverse kinematics (IK) of the Mitsubishi PA-10 7-DoF robot arm are solved in an analytical manner, in order for the human's and the robot artifact's end-effectors to achieve same position and orientation in 3D space (functional constraint). Redundancy is handled in the solution space of the robot arm's IK, selecting the most anthropomorphic solution computed, with a criterion of “Functional Anthropomorphism”. Human hand motion is transformed to robot hand motion using the joint-to-joint mapping methodology. Finally in order for the user to be able to detect contact and “perceive” the forces exerted by the robot hand, a low-cost force feedback device, that provides a mixture of sensory information (visual and vibrotactile), was developed.

A learning scheme based on random forests is used to discriminate between different reach to grasp movements in 3-D space, based on the myoelectric activity of human muscles of the upper-arm and the forearm. Task specificity for motion decoding is introduced in two different levels: Subspace to move toward and object to be grasped. The discrimination between the different reach to grasp strategies is accomplished with machine learning techniques for classification. The classification decision is then used in order to trigger an EMG-based task-specific motion decoding model. Task specific models manage to outperform "general" models providing better estimation accuracy. Thus, the proposed scheme takes advantage of a framework incorporating both a classifier and a regressor that cooperate advantageously in order to split the task space. The proposed learning scheme can be easily used to a series of EMG-based interfaces that must operate in real time, providing data-driven capabilities for multiclass problems, that occur in everyday life complex environments.

Reaching and grasping of objects in an everyday-life environment seems so simple for humans, though so complicated from an engineering point of view. Humans use a variety of strategies for reaching and grasping anything from the simplest to the most complicated objects, achieving high dexterity and efficiency. This seemingly simple process of reach-to-grasp relies on the complex coordination of the musculoskeletal system of the upper limbs. In this paper, we study the muscular co-activation patterns during a variety of reach-to-grasp motions, and we introduce a learning scheme that can discriminate between different strategies. This scheme can then classify reach-to-grasp strategies based on the muscular co-activations. We consider the arm and hand as a whole system, therefore we use surface ElectroMyoGraphic (sEMG) recordings from muscles of both the upper arm and the forearm. The proposed scheme is tested in extensive paradigms proving its efficiency, while it can be used as a switching mechanism for task-specific motion and force estimation models, improving EMG-based control of robotic arm-hand systems.

In this paper we propose a generic methodology for human to robot motion mapping for the case of a robotic arm hand system, allowing anthropomorphism. For doing so we discriminate between Functional Anthropomorphism and Perceptional Anthropomorphism, focusing on the first to achieve anthropomorphic solutions of the inverse kinematics for a redundant robot arm. Regarding hand motion mapping, a “wrist” (end-effector) offset to compensate for differences between human and robot hand dimensions is applied and the fingertips mapping methodology is used. Two different mapping scenarios are also examined: mapping for teleoperation and mapping for autonomous operation. The proposed methodology can be applied to a variety of human robot interaction applications, that require a special focus on anthropomorphism.

Coupling the human upper limbs with robotic devices is gaining increasing attention in the last decade, due to the emerging applications in orthotics, prosthetics and rehabilitation devices. In the cases of every-day life tasks, force exertion and generally interaction with the environment is absolutely critical. Therefore, the decoding of the user's force exertion intention is important for the robust control of orthotic robots (e.g. arm exoskeletons). In this paper, the human arm manipulability is analyzed and its effect on the recruitment of the musculo-skeletal system is explored. It was found that the recruitment and activation of muscles is strongly affected by arm manipulability. Based on this finding, a decoding method is built in order to estimate force exerted in the three-dimensional (3D) task space from surface ElectroMyoGraphic (EMG) signals, recorded from muscles of the arm. The method is using the manipulability information for the given force task. Experimental results were verified in various arm configurations with two subjects.

Humans perform a wide range of skillful and dexterous motion by adjusting the dynamic characteristics of their musculoskeletal system during motion. This capability is based on the non-linear characteristics of the muscles and the motor control architecture that can control motion and exerted force independently. Mechanical impedance (i.e. stiffness, viscosity and inertia) constitutes the most solid characteristic for describing the dynamic behavior of human movements. This paper presents a method for estimating upper limb impedance characteristics in the three-dimensional (3D) space, covering a wide range of the arm workspace. While subjects maintained postures, a seven-degrees-of-freedom (7-DoFs) robot arm was used to produce small displacements of subjects' hands along the three Cartesian axes. The end-point dynamic behavior was modeled using a linear second-order system and the impedance characteristics in the 3D space were identified using the measured forces and motion profiles. Experimental results were confirmed with two subjects.