Unmanned air vehicles (UAVs) have traditionally been considered as "eyes in the sky", that can move in three dimensions and need to avoid any contact with their environment. On the contrary, contact should not be considered as a problem, but as an opportunity to expand the range of UAVs applications. In this paper, we designed, fabricated, and characterized a whisker sensor unit based on MEMS barometers suitable for tactile localization on UAVs, featuring lightweight, low stiffness, high sensitivity, a broad sensing range, and scalability. Then, for the challenging task of contact point localization, we propose a Recurrent Multi-output Network (RMN) for predicting 3D contact points under continuous contact conditions to address the problems of non-linearity, hysteresis, and non-injective mapping between signals and contact points by considering time series. In addition, we propose an azimuth prediction loss function which reduces the RMSE by 3.24◦compared to L1loss. Finally, we conduct experiments on a linear stage to validate the 3D contact point localization capability of the proposed whisker system and model. The results show that our localization can achieve excellent performance, with an inference time of 1.4 ms and a mean error of only 9.18 mm in Euclidean distance within 3D space, laying a robust foundation for future implementation of tactile localization on UAVs. The design files, dataset, and source code are available on: https://github.com/BioMorphic-Intelligence-Lab/Whisker-3D-Localization.

Multi-sensor based motion tracking is of great interest to the robotics community as it may lessen the need for expensive optical motion capture equipment. However, the traditional convolution algorithms have difficulty adapting to the data due to the changes of joints’ relative position during motion. The time-series networks often used in the past ignore the spatial characteristics of sensors. We tackle this challenge by combining long short-term memory (LSTM) with graph convolution network (GCN), adding the prior knowledge of sensor distribution, and integrating it into the motion law through the adjacency matrix. This article proposes a novel shortcut enhanced LSTM-GCN network (SE-LSTM-GCN). It connects LSTM and GCN in sequence and extracts temporal and spatial features of data. At the same time, the shortcut is used in the network to enhance the output of two middle layers and to restore the filtered information. Our experimental results on two different motion tracking datasets show that the proposed network is able to learn the mapping relationship with better universality, less tracking error, and without increasing much training time, and can better perform human motion tracking tasks.Note to Practitioners—Accurate and real-time multiple soft sensors motion tracking suits are more accepted for their low cost. However, the soft-sensor based motion tracking is not comparable to the traditional optical equipment in prediction error. To this end, we present a novel network shortcut enhanced LSTM-GCN (SE-LSTM-GCN), consisting of shortcuts, long short-term memory (LSTM), and graph convolution network (GCN). The LSTM solves the non-linear and hysteresis of soft strain sensors, and GCN is integrated into the network since the knowledge of sensor location can be put into the adjacency matrix generated by the k-nearest neighbor (KNN). While shortcuts are used to enhance the output of middle layers to form combined features. Experimental results on two public datasets show that the proposed network is superior to competing algorithms in terms of prediction error. The network can be deployed in embedded devices, such as VR gloves to provide a better gaming experience. The current algorithm is based on the relationship between sensor data and distance. In future research, we will focus on adding other human kinematics laws to the network.

Hardness is one of the most critical tactile properties for robots to recognize objects. Machine learning methods have shown superior performance in object hardness classification. However, existing machine learning methods for tactile hardness classification cannot use the ordinal information between hardness classes because the one-hot encoding only cares about the correct class and ignores the inter-class relationship. To solve this problem, we propose to generalize the one-hot encoding using unimodal distributions including the Poisson and binomial distributions for tactile ordinal classification problems, resulting in two tactile ordinal networks (TacONet): TacONet-p and TacONet-b. Furthermore, we collect a tactile hardness dataset on the silicone samples with three different shapes (Shapes A, B, C), and each shape samples have thirteen hardness classes ranging from 0A (Shore A scale) to 60A at 5A intervals. We validate the resulting method for tactile hardness classification using a real robot. Experimental results demonstrate that compared with state-of-the-art methods, the proposed method achieves better classification performance in terms of accuracy and quadratic weighted kappa (QWK) on the tactile hardness dataset, reaching a classification accuracy up to 99.5% and a QWK up to 99.9% on Shape C. Note to Practitioners—In the field of robotics tactile recognition, hardness classification is one of the most important and common tasks for robots to accurately recognize objects, particularly when the environment is dark or visual sensors are not working. In this paper, we propose a novel tactile ordinal network for tactile hardness classification tasks. The existing machine learning models for tactile hardness classification are trained by minimizing the cross-entropy loss between predicted vectors and one-hot encoding vectors of true classes, which makes the models only care about the correct classes and ignores the inter-class relationship of hardness classes. In other words, these models have the same probability to misclassify a hardness class with any other hardness class. To tackle this problem, we propose to generalize the one-hot encoding method using a unimodal distribution method to encode the true classes. The unimodal distribution encoding vectors can make the model learn the ordinal information between classes. It is proved that the proposed method is able to effectively improve the classification accuracy and QWK in a tactile hardness classification task.