Skill transfer from humans to robots is challenging. Presently, many researchers focus on capturing only position or joint angle data from humans to teach the robots. Even though this approach has yielded impressive results for grasping applications, reconstructing motion for object handling or fine manipulation from a human hand to a robot hand has been sparsely explored. Humans use tactile feedback to adjust their motion to various objects, but capturing and reproducing the applied forces is an open research question. In this paper we introduce a wearable fingertip tactile sensor, which captures the distributed 3-axis force vectors on the fingertip. The fingertip tactile sensor is interchangeable between the human hand and the robot hand, meaning that it can also be assembled to fit on a robot hand such as the Allegro hand. This paper presents the structural aspects of the sensor as well as the methodology and approach used to design, manufacture, and calibrate the sensor. The sensor is able to measure forces accurately with a mean absolute error of 0.21, 0.16, and 0.44 Newtons in X, Y, and Z directions, respectively.

Detecting the onset/ongoing of slip, i.e. if a grasped object is slipping or will slip from the gripper while being lifted, is crucial. Conventionally, it is regarded as a tactile sensing related problem. However, recently multi-modal robotic learning has become popular and is expected to boost the performance. In this paper we propose a novel CNN-TCN model to fuse tactile and visual information for detecting the onset/ongoing of slip. In our experiments, two uSkin tactile sensors and one Realsense435i camera are used. Data is collected by randomly grasping and lifting 35 daily objects 1050 times in total. Furthermore, we compare our CNN-TCN model with the widely used CNN-LSTM model. As a result, our proposed model achieves a 88.75% detection accuracy and outperforms the CNN-LSTM model combined with different pretrained vision networks.

Recently, tactile sensing has attracted great interest for robotic manipulation. Predicting if a grasp will be stable or not, i.e. if the grasped object will drop out of the gripper while being lifted, can aid robust robotic grasping. Previous methods paid equal attention to all regions of the tactile data matrix or all time-steps in the tactile sequence, which may include irrelevant or redundant information. In this paper, we propose to equip Convolutional Neural Networks with spatial-channel and temporal attention mechanisms (SCT attention CNN) to predict future grasp stability. To the best of our knowledge, this is the first time to use attention mechanisms for predicting grasp stability only relying on tactile information. We implement our experiments with 52 daily objects. Moreover, we compare different spatio-temporal models and attention mechanisms as an empirical study. We found a significant accuracy improvement of up to 5% when using SCT attention. We believe that attention mechanisms can also improve the performance of other tactile learning tasks in the future, such as slip detection and hardness perception.

This paper presents the design and evaluation of the low-cost capacitive proximity-force-fusion sensor "safe skin", which can measure simultaneously the proximity of humans as well as the contact force. It was designed such that the force and proximity sensing functions can work concurrently without interfering with each other. Moreover, active shielding, on-chip digitization and ground isolation are implemented for the sensor to minimize the influence from stray capacitance and electromagnetic interference (EMI) from the environment, which ensures that the sensor has a high system robustness for industrial applications. The prototype version has the capability of detecting a grounded human hand sized object from a distance of 400 mm. Moreover, forces in the range of 5 to 40 N can be measured, with 43.7% hysteresis and 6.7% nonlinearity. Due to its sensing characteristics, when used on a robot, the sensor could be used to ensure the safety of nearby humans in the future. The sensor could also potentially be used as an interface for human-robot interaction (HRI).

Measuring the forces that humans exert with their fingers could have many potential applications, such as skill transfer from human experts to robots or monitoring humans. In this paper we introduce the "Exo-Glove" system, which can measure the joint angles and forces acting on the human finger without covering the skin that is in contact with the manipulated object. In particular, 3-axis sensors measure the deformation of the human skin on the sides of the finger to indirectly measure the 3-axis forces acting on the finger. To provide a frame of reference for the sensors, and to measure the joint angles of the human finger, an exoskeleton with remote center of motion (RCM) joints is used. Experiments showed that with the exoskeleton the quality of the force measurements can be improved.

The sense of touch is often crucial for humans to perform manipulation tasks. Providing tactile feedback during teleoperation or for users of prosthetic devices would be beneficial. However, the representation of tactile information constitutes a major technical challenge, since the numerous and possibly multimodal sensor readings are massive compared to the available tactile display technology. We introduce an algorithm that deploys two stages of K-means clustering along and across tactile image frames that render tactile sensor information at each time instant. In this manner, the massive tactile information is adaptively compressed in real-time while preserving its physical meaning, thus, remains intuitive and direct. We experimentally verify and examine the characteristics of our algorithm by evaluating the original and compressed tactile data. The data was gathered during the active tactile exploration of several objects of daily living by an Allegro robot hand that was covered with 15 uSkin sensor modules providing 2403-axis force vector measurements at each time instant. Our novel algorithm is straight forward enough to be implemented into tactile feedback systems. Finally, our algorithm allows for the direct feedback of massive tactile sensor data for a broad variety of tactile sensors and tactile displays, thereby, enables the compressed yet intuitive representation of massive tactile sensor information for real-time applications.

This paper investigates tactile object recognition with relatively densely distributed force vector measurements and evaluates what kind of tactile information is beneficial for object recognition. The uSkin tactile sensors are embedded in an Allegro Hand, and provide 240 triaxial force vector measurements in total in all fingers. Active object sensing is used to gather time-series training and testing data. A simple feedforward, a recurrent, and a convolutional neural network are used for recognizing objects. Evaluations with different number of employed measurements, static vs. time series data and force vector vs. only normal force vector measurements show that the high-dimensional information provided by the sensors is indeed beneficial. An object recognition rate of up to 95% for 20 objects was achieved.

Typically, the range and sensitivity of force sensors are determined during production. However, to be able to do both delicate and high-force demanding work, adjustable force sensitivity would be beneficial. The current paper proposes such a sensor by implementing a planar electromagnet above a 3-axis magnetic sensor, separated by soft foam. Furthermore, the sensor has digital output with an integrated microcontroller. The magnetic field strength with varying currents is examined in simulation, and the field changes according to displacements are investigated both in simulation and with the actual sensor. A prototype 3-axis force sensor is implemented and the relationship between the magnetic field change and the corresponding applied force is also investigated. It could be shown that the sensitivity of the sensor to displacements, as well as force, can indeed be adjusted.

The advantages of mechanical compliance have driven the development of devices using new smart materials. A new kind of magnetorheological piston based on a toroidal array of magnetorheological valves, has been previously tested to prove its feasibility. However, being an initial prototype its potential was still limited by its complex design, and low output force. This study presents the revisions done to the design with several improvements targeting key performance parameters. An improved annular piston design is also introduced as comparison with conventional devices. The toroidal and annular piston head prototypes are built and tested, and their force performance compared with the previous iteration. The experimental results show an overall performance improvement of the toroidal assembly. However, the force model used in the study still fails to accurately predict the magnetic flux at the gaps of the piston head. This deviation is later verify and corrected using a FEM analysis. The force performance of the new toroidal assembly is on par with the commonplace annular design. It also displays a more linear behaviour, at the expense of lower energy efficiency. Finally, it also shows potential for a greater degree of customisation to meet different system requirements.

Moving objects within the hand is challenging, especially if the objects are of various shape and size. In this paper we use machine learning to learn in-hand manipulation of such various sized and shaped objects. The TWENDY-ONE hand is used, which has various properties that makes it well suited for in-hand manipulation: a high number of actuated joints, passive degrees of freedom and soft skin, six-axis force/torque (F/T) sensors in each fingertip, and distributed tactile sensors in the skin. A dataglove is used to gather training samples for teaching the required behavior. The object size information is extracted from the initial grasping posture. After training a neural network, the robot is able to manipulate objects of untrained sizes and shape. The results show the importance of size and tactile information. Compared to interpolation control, the adaptability for the initial posture gap could be greatly extended. Final results show that with deep learning the number of required training sets can be drastically reduced.