Currently, deep neural networks can be extremely effective in robotic tactile perception. However, a major challenge is to solve the problem of continual learning of robotic tactile perception in an open and dynamic environment. In this paper, we propose a novel continual learning method for the domian incremental learning task in the field of tactile perception. To be specific, we introduce a morphology-specific variational autoencoders which can mitigate catastrophic forgetting by generating pseudo-samples for training in the continual learning process. We integrate the generative model and the discriminative model into one model, which reduces the size of model and improves the continual learning ability. In addition, considering the ordinal information between the hardness levels, we propose to add conditional information to the model and introduce a modified loss function to combine the latent value with the hardness information, which improves the continual learning performance by controlling the distribution and quality of pseudo-sample generation. Following this, we designed a tactile robot experiment, collected hardness data, and tested our model on this object hardness recognition task. We show experimentally that, after training, the model can still maintain the accuracy of more than 94% after learning three tasks in terms. Note to Practitioners—In the field of robotics tactile perception, the issue of continual learning in robots is a crucial problem that urgently requires resolution. We hope robots to effectively engage in continual learning across multiple tasks, ensuring the acquisition of new knowledge while mitigating the risk of forgetting previously acquired knowledge. In this paper, we propose a novel continual learning method for the domian incremental learning task. we introduce a morphology-specific variational autoencoders based on replaying pseudo-samples during continual learning process which reduces the size of model and improves the continual learning ability. We enhance model performance by integrating generative and discriminative models, incorporating conditional information to control the distribution of replayed sample types, and leveraging sequential relationships among samples. It is proved that the proposed method is able to effectively improve the accuracy in a tactile domian incremental learning task.

Inadequate consideration of tactile sensor features in existing algorithms can lead to insufficient prediction accuracy for robotic tactile recognition tasks. In this paper, we propose a spatial-temporal information based adjacency matrix construction method called transfer cost. On this basis, we propose a multi-branch multi-scale channel fusion graph convolutional network (MSCF-LSTM-GCN). This network combines the advantages of long short-term memory networks (LSTM) with graph convolutional networks (GCN) to extract spatial-temporal features. It is the first LSTM-GCN network with a multi-branch structure that utilizes multi-feature scale and multi-channel fusion. This structure provides diverse perceptual ranges and information levels. We introduce shortcuts combined with the multi-branch structure to reduce the loss of information caused by multiple layers of transmission. The Einstein summation calculation method and the parameters of binarizing the adjacency matrix are optimized to enable the established dynamic tactile graph to participate in batch training. The proposed method improves both prediction accuracy and F1 score by over 1% on the tactile grasp stability prediction task. In the tactile object recognition task, the proposed method improves both prediction accuracy and F1 score by over 3%.Note to Practitioners—Robotic tactile perception is an important research area that can improve the flexibility, accuracy and stability of robots in robotic tactile recognition tasks. However, the amount of accessible information depends on the resolution and number of tactile sensors, and the features extracted using conventional deep learning methods are limited. For this reason, we propose a spatial-temporal information based adjacency matrix construction method called transfer cost. Then, we propose the MSCF-LSTM-GCN model that extract spatial-temporal features from different feature scales and shortcuts. The network solves the problems of traditional deep learning methods in acquiringsingle features and losing deep information. The experiments on two public datasets show that our method performs due to other competing algorithms on the tasks of robotic grasp stability prediction and object recognition for multi-tactile sensor scenarios. In future research we will invest in the problem of 3D force regression.

The sense of touch is essential for robots to perform various daily tasks. Artificial Neural Networks have shown significant promise in advancing robotic tactile learning. However, due to the changing of tactile data distribution as robots encounter new tasks, ANN-based robotic tactile learning suffers from catastrophic forgetting. To solve this problem, we introduce a novel continual learning (CL) framework called the Probabilistic Spiking Neural Network with Variational Continual Learning (PSNN-VCL). In this framework, PSNN introduces uncertainty during spike emission and can apply fast Variational Inference by optimizing the uncertainty through backpropagation, which significantly reduces the required model parameters for VCL. We establish a robotic tactile CL benchmark using publicly available datasets to evaluate our method. Experimental results demonstrated that, compared to other CL methods, PSNN-VCL not only achieves superior performance in terms of widely used CL metrics but also achieves at least a 50% reduction in model parameters on the robotic tactile CL benchmark.