We propose a neuromorphic tactile sensing frame-work for robotic texture classification that is inspired by human exploratory strategies. Our system utilizes the NeuroTac sensor to capture neuromorphic tactile data during a series of exploratory motions. We first tested six distinct motions for texture classification under fixed environment: sliding, rotating, tapping, as well as the combined motions: sliding+rotating, tapping+rotating, and tapping+sliding. We chose sliding and sliding+rotating as the best motions based on final accuracy and the sample timing length needed to reach converged accuracy. In the second experiment designed to simulate complex real-world conditions, these two motions were further evaluated under varying contact depth and speeds. Under these conditions, our framework attained the highest accuracy of 87.33% with sliding+rotating while maintaining an extremely low power consumption of only 8.04 mW. These results suggest that the sliding+rotating motion is the optimal exploratory strategy for neuromorphic tactile sensing deployment in texture classification tasks and holds significant promise for enhancing robotic environmental interaction.

Tactile exploration of surfaces is a key component of everyday life, allowing us to make complex inferences about our environments even when vision is occluded. The emergence of biomimetic neuromorphic hardware in recent years has furthered our ability to create biologically plausible sensing solutions. While these platforms continue to improve in regards to latency and power consumption, within recent literature on tactile texture classification there is an emphasis on accuracy at the expense of real-time processing. In order for these tactile sensing systems to find use outside of experimental laboratory environments, it is key to design systems capable of capturing and processing data in real-time. Within this paper we present a system for the real-time classification of texture using a neuromorphic tactile sensor, a spiking neural network and a novel decision making algorithm. Our real-time system achieves classification accuracies of 94% on a dataset of 11 natural textile textures. Furthermore our system is capable of identifying textures at human-level performance in as little as 84ms. Additionally, benchmarking our system across CPU, GPU and Loihi2 hardware platforms resulted in a 96% reduction in power consumption on the neuromorphic platform. This system out-performed previous work by the authors and the state of art, both in terms of accuracy and classification speed.

Incipient slip is defined as the slippage of part, but not all, of the contact surface between a sensor and an object. Reliably detecting incipient slip in artificial tactile sensors would benefit autonomous robot handling capabilities by helping prevent object slippage during manipulation. Here, we present a biomimetic skin morphology based on the human fingerprint with application to marker-based tactile sensors such as the TacTip biomimetic optical tactile sensor. We modify the 3D-printed outer membrane of the TacTip to mimic glabrous skin morphology with the inclusion of external ridges (fingerprint) and internal markers (intermediate ridges), allowing localised shear deformation of the sensor's skin prior to the onset of gross slip. To validate the performance of this skin morphology, we train a random forest classifier (RFC) to identify incipient slip based on the extracted marker displacements from the sensor when it is compressed against an acrylic plate and moved laterally. The RFC model achieves 97.46% accuracy on incipient slip prediction, and is then validated on an unseen pouring task, in which gravity-induced incipient slip is detected on average within$418\pm 753\ \text{ms}$of its onset, and before gross slip in all trials. This accurate detection of incipient slip enables corrective actions prior to the onset of gross slip, a key capability in robotic manipulation and upper-limb prosthetics.

Tactile feedback is necessary for closing the sen-sorimotor loop in prosthetic and tele-operable control, which would allow for more precise manipulation and increased acceptance of use of such devices. Pressure stimuli are commonly presented to users in haptic devices through a sensory substitution to vibration. The precise nature of this substitution affects pressure sensitivity, as well as the comfort and intuitiveness of the device for the user. This study determines the effects of different vibrational encodings for pressure on user-preference and performance in a 4-alternative absolute identification task. 4 different encoding patterns for pressure were examined: short pulse and long pulse amplitude modulation along with sine and square wave frequency modulation. Of the methods examined, frequency modulation methods led to the best discrimination of stimuli (p$p$= 0.098). This suggests that prostheses or teleoperated devices utilising vibrotactile feedback may benefit from implementing a discrete frequency-based sinusoidal pattern to indicate changes in grip force.

In this work, we report on how a sense of touch can be used to control an underactuated anthropomorphic robot hand, based on an integration that respects the hand’s mechanical functionality. Our focus is on integrating the sensorimotor control of the Pisa/IIT SoftHand, an anthropomorphic soft robot hand designed around the principle of adaptive synergies, with the BRL tactile fingertip (TacTip), a soft biomimetic optical tactile sensor. We consider: (i) closed-loop tactile control to establish a light contact on an unknown held object, based on the structural similarity of the tactile image; and (ii) controlling the estimated pose of a held object, using a convolutional neural network approach developed for other TacTip sensors. Accurate control was found for a range of hard and soft objects (to sub-millimetre accuracy and a few degrees). Overall, this gives a foundation to endow soft robotic hands with human-like touch, with implications for autonomous grasping, manipulation, human-robot interaction and prosthetics.

Developing artificial tactile sensing capabilities that rival human touch is a long-term goal in robotics and prosthetics. Gradually more elaborate biomimetic tactile sensors are being developed and applied to grasping and manipulation tasks to help achieve this goal. Here we present the neuroTac, a novel neuromorphic optical tactile sensor. The neuroTac combines the biomimetic hardware design from the TacTip sensor which mimicks the layered papillae structure of human glabrous skin, with an event-based camera (DAVIS240, iniVation) and algorithms which transduce contact information in the form of spike trains. The performance of the sensor is evaluated on a texture classification task, with four spike coding methods being implemented and compared: Intensive, Spatial, Temporal and Spatiotemporal. We found timing-based coding methods performed with the highest accuracy over both artificial and natural textures. The spike-based output of the neuroTac could enable the development of biomimetic tactile perception algorithms in robotics as well as non-invasive and invasive haptic feedback methods in prosthetics.

Restoring tactile sensation is essential to enable in-hand manipulation and the smooth, natural control of upper-limb prosthetic devices. Here we present a platform to contribute to that long-term vision, combining an anthropomorphic robot hand (QB SoftHand) with a neuromorphic optical tactile sensor (neuroTac). Neuromorphic sensors aim to produce efficient, spike-based representations of information for bio-inspired processing. The development of this 5-fingered, sensorized hardware platform is validated with a customized mount allowing manual control of the hand. The platform is demonstrated to succesfully identify 4 objects from the YCB object set, and accurately discriminate between 4 directions of shear during stable grasps. This platform could lead to wide-ranging developments in the areas of haptics, prosthetics and telerobotics.

Tactile manipulation is the ability to control objects in real-time using the sense of touch. Here we examine tactile manipulation from the perspective of active touch with a biomimetic tactile sensor, which combines tactile perception with control of sensor location. Experiments are performed with the tactile fingertip mounted as end effector to a robot arm, to manipulate (roll) a cylinder in contact with the fingertip. Performance is validated with offline (cross-validation) and online (real-time operation) assessments. Location perception is finer than the sensor resolution, leading to superresolved tactile manipulation along a complex trajectory. However, the original methods were non-robust to large unknown disturbances of object location, necessitating modification of the perceptual process to diminish prior beliefs relative to past posterior beliefs. In consequence robust and accurate tactile manipulation was attained. In general, it appears there is a trade-off between the responsiveness to unknown change and manipulation accuracy, which must be set appropriately for each task.

Although superresolution has been studied to huge impact in visual imaging, it is relatively unexplored in tactile robotics. Here we demonstrate a novel optical sensor design (the TacTip) capable of achieving 40-fold localization superresolution to 0.1mm accuracy compared with a 4mm resolution between tactile elements. This superresolution is reached for localizing a 40mm diameter hemicylinder with a tactile finger pad also of 40mm diameter. Deformations of the sensor surface are measured as displacements of molded internal pins, with pin separation thus defining sensor resolution. Active Bayesian perception for classifying object location was used to ensure robust localization and hence the magnitude of the superresolution. These results are comparable with those for capacitive tactile sensors, which we interpret as originating from a convergence in the taxel-based design of the optical sensor and capacitive tactile sensors. The attained superresolution is comparable to the best perceptual hyperacuity in humans.