The supernumerary robotic limb (SRL) mounted on the shoulder has been demonstrated to be able to serve as a third arm to assist human in overhead support task. However, the mechanical connection between the wearer and the SRL means that human operator’s movement will continually disturb the SRL and may lead to instability. Moreover, there may be physical conflicts between SRL and human operator due to their different intentions, which may potentially increase the load on human operator. Therefore, it’s necessary to control SRL to ensure stable supporting and transparent interaction. Here, we propose an adaptive controller for human movement compensation. Firstly, we model the human-SRL coordinative behavior based on non-zero-sum differential game theory aiming to enhance the support stability and reduce the operator’s load, which is a framework capable of dynamically regulating the control strategies between two interacting agents. We then implement an adaptive control strategy that adjusts the SRL’s input optimally responsive to the human operator’s input in the sense of Nash equilibrium to meet predefined control objectives. From experimental results during overhead support task, the proposed controller reduces the peak reaction force from 15.59 N to 4.50 N compared to the state-of-the-art controller that disregards human input while ensuring the stability of support, thereby providing advantage for human-SRL coordination in overhead support task.

As virtual objects contain increasingly rich attribute information, small wearable fingertip devices need to have higher degrees of freedom (DoFs) to convey the haptic sensation of virtual objects. In order to effectively display the shape features of virtual objects to users through curvature, we designed a 6-DoF wearable fingertip device (WFD). This WFD combines a 6-DoF Stewart parallel mechanism, consisting of a static platform and a mobile platform connected by six revolute-spherical-spherical kinematic chains. The translation and rotation of the mobile platform are driven by six miniature servo motors, which can simulate haptic sensations such as making and breaking contact, sliding, and skin stretch when the fingertip interacts with a virtual surface. The WFD is fixed at the user's dominant index finger using hook-and-loop fasteners, with a size of 68 × 59 × 56 mm$^{3}$3 and a mass of 45.5 g. We analyzed and validated the kinematic model of the WFD and tested its force output capability. Finally, we invited 15 adults to conduct three subjective perception experiments to evaluate the performance of the WFD in curvature perception and shape display. The experimental results show that: (1) The just noticeable difference (JND) for curvature identification using the WFD is 3.02$\pm$±0.23 m$^{-1}$-1; (2) The 6-DoF haptic feedback provided by the WFD improves the accuracy of curved surface recognition from 53.4$\pm$±7.1% in 3-DoF to 72.0$\pm$±5.9%; (3) Even without visual feedback, the shape recognition accuracy of the WFD when combined with the Touch device reaches 82.3$\pm$±8.2% . Experimental results show that the WFD has good performance and potential in curvature perception and shape display.

Robot-assisted eye-hand coordination rehabilitation training system is extremely urgent to study since recent evidence suggests that eye-hand coordination can be brutally disturbed by stroke with critical consequences on motor behavior. In this paper, we develop a robot-assisted eye-hand coordination training system by estimating motion direction using smooth-pursuit eye movements. Firstly, we design a Pong Game, which requires users to extrapolate the direction of a linearly moving ball and to predict whether this ball would be hit. Secondly, the motion direction of the ball is estimated via smooth-pursuit eye movements, allowing the robot quickly establish an assistive force field to hit the ball. Thirdly, adding haptic feedback technology into this training system to make users more immersive. Finally, we conduct a feasibility study with eight healthy subjects to verify the effectiveness of the proposed system. The experimental results show that the mean success rate for hitting the pong ball of the experiment group (assistance turn-on) is 28.33% higher than that of the control group (assistance turn-off), and the mean interception time of the experiment group is 0.35s shorter than that of the control group. Therefore, the developed system may be promising for transferring to the robot-assisted eye-hand coordination rehabilitation training for post-stroke patients.

In this paper an automated data labeling (ADL) neural network is proposed to streamline dataset collecting for real-time predicting the continuous motion of hand and wrist, these gestures are only decoded from a surface electromyography (sEMG) array of eight channels. Unlike collecting both the bio-signals and hand motion signals as samples and labels in supervised learning, this algorithm only collects unlabeled sEMG into an unsupervised neural network, in which the hand motion labels are auto-generated. The coefficient of determination (R2) for three DOFs, i.e. wrist flex/extension, wrist pro/supination, hand open/close, was 0.86, 0.89 and 0.87 respectively. The comparison between real motion labels and auto-generated labels shows that the latter has earlier response than former. The results of Fitts’ law test indicate that ADL has capability of controlling multi-DOFs simultaneously even though the training set only contains sEMG data from single DOF gesture. Moreover, no more hand motion measurement needed which greatly helps upper limb amputee imagine the gesture of residual limb to control a dexterous prosthesis.

The brain-machine interface (BMI) has been reported to offer the potential for controlling the assistive robot for the motor impaired people, using the non-invasively obtained electroencephalogram (EEG) signals. However, the EEG based BMI may not be sufficient and stable to drive the robot moving freely in its 2D or 3D workspace. The robot autonomy may provide assistance for the BMI users with the shared control paradigm. Nevertheless, users suffers from several limitations of the current shared control paradigms applied on BMI, e.g., loss of sense of control, high mental workload due to unintuitive control with the human-robot interface and fixed level of assistance. To overcome these drawbacks, we propose a new control paradigm for the robotic arm reaching task where the robot autonomy is dynamically blended with the gaze-BMI control from a user. In this paradigm, the hybrid gaze-BMI constitutes an intuitive and effective input to continuously control the robotic arm end-effector moving freely in its 2D workspace, with an adjustable speed proportional to the motion intention strength. Furthermore, the adjustable level of assistance by our paradigm allows the system to balance the user's capabilities and feelings of control while compensating for the reaching task's difficulty. The proposed paradigm is verified in the task where a healthy subject utilizes the hybrid gaze-BMI to control the robotic arm end-effector reaching for a target object while avoiding the obstacle in the path. The experimental results demonstrate that the movements with our shared control paradigm are safer, more efficient and less difficult than those without shared control.

The utility to decode hand movement parameters is significant to the control of artificial limb in the BCI fields. Most previous studies have adopted amplitude features of the low-frequency EEG signals to decode hand movement parameters. In this study, we have investigated the instantaneous phase of the low-frequency EEG signals attained by Hilbert transform for such a task for the first time, and compared its decoding accuracy with that of the amplitude features. An experiment was carried out that 5 subjects executed a center-out reaching task in two sessions. Then the Multiple Linear Regression (MLR) model is used to decode hand movement parameters based on the amplitude feature and the phase feature, respectively. The performance of the proposed approach is evaluated by calculating the correlation coefficients between the recorded parameters and the reconstructed parameters. The experiments results show that compared to the decoder with the amplitude feature, the correlation coefficients obtained by the decoder with the phase feature have increased 27.8% (X-position), 24.1% (Y-position), 27.9% (X-velocity), 20.9% (Y-velocity).

In addition to the noisy and limited spatial resolution characteristics of the electroencephalography (EEG) signal, the intrinsic nonstationarity in the EEG data makes the single-trial EEG classification an even more challenging problem in brain-computer interface (BCI). Variations of the signal properties within a session often result in deteriorated classification performance. This is mainly attributed to the reason that the routine feature extraction or classification method does not take the changes in the signal into account. Although several extensions to the standard feature extraction method have been proposed to reduce the sensitivity to nonstationarity in data, they optimize different objective functions from that of the subsequent classification model, and thereby, the extracted features may not be optimized for the classification. In this paper, we propose an approach that directly optimizes the classifier's discriminativity and robustness against the within-session nonstationarity of the EEG data through a single optimization paradigm, and show that it can greatly improve the performance, in particular for the subjects who have difficulty in controlling a BCI. Moreover, the experimental results on two benchmark data sets demonstrate that our approach significantly outperforms the compared approaches in reducing classification error rates.

The pairwise constraints specifying whether a pair of samples should be grouped together or not have been successfully incorporated into the conventional clustering methods such as k-means and spectral clustering for the performance enhancement. Nevertheless, the issue of pairwise constraints has not been well studied in the recently proposed maximum margin clustering (MMC), which extends the maximum margin framework in supervised learning for clustering and often shows a promising performance. This paper therefore proposes a pairwise constrained MMC algorithm. Based on the maximum margin idea in MMC, we propose a set of effective loss functions for discouraging the violation of given pairwise constraints. For the resulting optimization problem, we show that the original nonconvex problem in our approach can be decomposed into a sequence of convex quadratic program problems via constrained concave-convex procedure (CCCP). Subsequently, we present an efficient subgradient projection optimization method to solve each convex problem in the CCCP sequence. Experiments on a number of real-world data sets show that the proposed constrained MMC algorithm is scalable and outperforms the existing constrained MMC approach as well as the typical semi-supervised clustering counterparts.

The performance of the most clustering algorithms highly relies on the representation of data in the input space or the Hilbert space of kernel methods. This paper is to obtain an appropriate data representation through feature selection or kernel learning within the framework of the Local Learning-Based Clustering (LLC) (Wu and Schölkopf 2006) method, which can outperform the global learning-based ones when dealing with the high-dimensional data lying on manifold. Specifically, we associate a weight to each feature or kernel and incorporate it into the built-in regularization of the LLC algorithm to take into account the relevance of each feature or kernel for the clustering. Accordingly, the weights are estimated iteratively in the clustering process. We show that the resulting weighted regularization with an additional constraint on the weights is equivalent to a known sparse-promoting penalty. Hence, the weights of those irrelevant features or kernels can be shrunk toward zero. Extensive experiments show the efficacy of the proposed methods on the benchmark data sets.

In general, irrelevant features of high-dimensional data will degrade the performance of an inference system, e.g., a clustering algorithm or a classifier. In this paper, we therefore present a Local Kernel Regression (LKR) scoring approach to evaluate the relevancy of features based on their capabilities of keeping the local configuration in a small patch of data. Accordingly, a score index featuring applicability to both of supervised learning and unsupervised learning is developed to identify the relevant features within the framework of local kernel regression. Experimental results show the efficacy of the proposed approach in comparison with the existing methods.

This paper proposes an algorithm to deal with the feature selection in Gaussian mixture clustering by an iterative way: the algorithm iterates between the clustering and the unsupervised feature selection. First, we propose a quantitative measurement of the feature relevance with respect to the clustering. Then, we design the corresponding feature selection scheme and integrate it into the rival penalized EM (RPEM) clustering algorithm (Cheung, 2005) that is able to determine the number of clusters automatically. Subsequently, the clustering can be performed in an appropriate feature subset by gradually eliminating the irrelevant features with automatic model selection. Compared to the existing methods, the numerical experiments have shown the efficacy of the proposed algorithm on the synthetic and real world data.