Robots can often learn skills from human demonstrations. Robot operations via visual perception are commonly influenced by the external environment and are relatively demanding in terms of external conditions, manual segmentation of tasks is time-consuming and labor-intensive, and robots do not perform complex tasks with sufficient accuracy and naturalness in their movements. In this work, we propose a programming by demonstration framework to facilitate autonomous task segmentation and flexible execution. We acquire surface electromyography (sEMG) signals from the forearm and train the gesture datasets by transfer learning from the sign language dataset to achieve action classification. Meanwhile, the inertial information of the forearm is collected and combined with the SEMG signals to autonomously segment operational skills into discrete task units, and after comparing with ground truth, it is demonstrated that multi-modal information can lead to higher segmentation accuracy. To make the robot movements more natural, we add arm stiffness information to this system and estimate the arm stiffness of different individuals by creating a muscle force map of the demonstrator. Finally, human manipulation skills are mapped onto the UR5e robot to validate the results of human-robot skill transfer.

In brain-computer interface (BCI) systems, symmetric positive definite (SPD) manifold within Riemannian space has been frequently utilized to extract spatial features from electroencephalogram (EEG) signals. However, the intrinsic high dimensionality of SPD matrices introduces too much computational burden to hinder the real-time applications of such BCI, especially in handling dynamic tasks, like incremental learning. Directly reducing the dimensionality of SPD matrices with conventional dimensionality reduction (DR) methods will alter the fundamental properties of SPD matrices. Moreover, current DR methods for incremental learning always necessitate retaining old data to update their representations under new mapping. To this end, a bidirectional two-dimensional principal component analysis for SPD manifold (B2DPCA-SPD) is proposed to reduce the dimensionality of SPD matrices, in such way that the reduced matrices remain on SPD manifold. Afterwards, the B2DPCA-SPD is extended to adapt to incremental learning tasks without saving old data. The incremental B2DPCA-SPD can be seamlessly integrated with the matrix-formed growing neural gas network (MF-GNG) to achieve an incremental EEG classification, where the new low-dimensional representations of the prototypes in old classifiers can be easily recalculated with the updated projection matrix. Extensive experiments are conducted on two public datasets to perform the EEG classification. The results demonstrate that our method significantly reduces computation time by 38.53% and 35.96%, and outperforms conventional methods in classification accuracy by 4.21% to 19.59%.

A novel convolution architecture PatchCNN is proposed for extending 2-D grid convolution to the nongrid structured data: point clouds, without any intermediate data representation. Previous studies implicitly capture local shape pattern from the meaningful subset or a local region without considering the interaction among points of the local region. The PointPatch module in our deep network, in spirit to the 8-pixels neighborhood in the 2-D image, explicitly models geometric relationship among points in the local region. We adopt a light 3-D convolution network to adaptively integrate features of the PointPatch module. The integrated features encode geometric relationship and the impact of surrounding points, which brings sufficient shape awareness and robustness for point cloud perception. Additionally, in our work, the convolution weight on each point is treated as a Lipschitz continuous function approximated by multilayer perceptron (MLP) and integrated features in the local region. Theoretically, the explicit learning strategy proposed in PatchCNN introduces inductive bias beneficial to the learning shape pattern in 3-D Euclidean space. Extensive experiments on ModelNet40 and ScanNet v2 data sets demonstrate that the proposed method achieves the competitive performance on par or even better than state-of-the-art methods.

In recent years, brain-controlled intelligent wheelchairs have received extensive attention, which combines the accessibility of the Brain-computer Interface (BCI) system with the intelligence of wheelchairs. However, current brain-controlled wheelchairs are always operated in a fixed mode. The Electroencephalogram (EEG) signals with the fixed acquisition time are analyzed without considering the state of the user, which not only increases the risk of misoperation, but seriously reduces the information transfer rate of the system. To solve this problem, an adaptive control approach for intelligent wheelchair based on BCI combining with Quality of Operating (QoO) is proposed. Firstly, the influence of motor imagery signals with different time lengths in different states on classification accuracy was analyzed using tangent space Support Vector Machine (TSSVM) algorithm. Then, the definition of QoO was introduced, which was obtained by analyzing sample entropy and power spectral density (PSD) of four kinds of EEG activities, delta, theta, alpha and beta. Finally, the acquisition time of required EEG signals was adjusted according to the value of QoO. We constructed a brain-controlled wheelchair system and conducted real environmental experiments for 9 subjects using strategies, with and without adaptive control approach. The results show that the approach proposed in this paper can reduce the risk of misoperation and increase the information transfer rate on the premise of ensuring the classification performance during navigation in complex indoor environment.