A novel collaborative position and orientation control scheme (CPOCS) for dual-arm robots is proposed, which is capable of controlling the end-effectors' positions with high precision while preserving their orientations unchanged to some practical tasks (e.g., box handling). To solve the proposed CPOCS in real time while considering key factors such as unknown bounded noise and strict time response constraints in practical engineering environments, this article proposes a novel precisely predefined-time convergent barrier recurrent neural network (PCB-RNN) based on a newly developed piecewise barrier evolution formula. Unlike existing RNNs, the proposed PCB-RNN, owing to its piecewise barrier evolution formula, can achieve precisely predefined-time convergence (PPTC) when addressing the proposed CPOCS under unknown bounded noise conditions. Comprehensive theoretical analysis rigorously proves the PPTC ability of the PCB-RNN under both noise-free and unknown bounded noise conditions. Furthermore, extensive simulation and physical experiments on dual-arm robots validate the effectiveness of the proposed CPOCS and demonstrate the advanced PPTC capability of the proposed PCB-RNN under unknown bounded noises.

Time-variant-gain zeroing neural networks (TVG-ZNNs) are among the most powerful solvers for time-variant linear matrix-vector inequalities (TVLMVIs). Although TVG-ZNNs with complex nonlinear activation functions achieve effective convergence within finite or predefined time, they incur high computational costs and face challenges in precisely predefining their actual convergence time. In contrast, TVG-ZNNs with linear activation functions offer lower computational costs but struggle to achieve convergence within a finite or predefined time. In addition, the gain values of most existing TVG-ZNNs tend to increase over time, resulting in a significant rise in computational costs. To address these contradictory issues, we propose a novel hybrid-gain ZNN without a nonlinear activation function (HG-ZNN-WNAF) to solve TVLMVIs in both noisy and noise-free environments. Specifically, a new hybrid gain is cleverly designed to construct the HG-ZNN-WNAF activated by a linear activation function, while ensuring that the gain value does not keep increasing over time. Unlike the state-of-the-art TVG-ZNNs with or without nonlinear activation functions, our proposed HG-ZNN-WNAF achieves precisely predefined-time convergence due to the hybrid gain, meaning its actual convergence time can be accurately predefined. Additionally, the piecewise design of the hybrid gain, along with the use of the simple linear activation function, effectively reduces the model's computational cost. Rigorous theoretical analysis demonstrates the precisely predefined-time convergence ability of the HG-ZNN-WNAF in both noisy and noise-free environments. Simulation and physical experiments validate the theoretical analysis and demonstrate that the HG-ZNN-WNAF achieves state-of-the-art performance in terms of convergence speed, robustness, and computational cost.

Zeroing neural network (ZNN), as a special type of recurrent neural network (RNN), is very competitive in solving time-varying linear matrix-vector equations. Recently, various ZNNs with predefined-time convergence (PTC) capabilities have been reported. Such ZNNs with PTC capabilities can achieve the predefined convergence time via explicitly presetting multiple parameters related to the upper bounds of their convergence time. However, obtaining suitable and robust values for these parameters through reasonable adjustments is a challenging task in many engineering applications. To address this problem, we propose a novel arbitrarily predefined-time convergent RNN (APTC-RNN) with a novel nonlinear piecewise activation-function (NPAF). Unlike most existing ZNNs with PTC capabilities, the proposed APTC-RNN, due to its NPAF, can achieve arbitrarily PTC (APTC) without adjusting any upper bound parameters. Furthermore, due to the piecewise computation form of the NPAF, the proposed APTC-RNN can provide a lower computational cost compared to most existing RNNs. The stability and APTC capability of the proposed APTC-RNN are proven by rigorous theoretical analysis and mathematical derivation. Numerical simulations show that APTC-RNN has faster and more accurate PTC capability than three state-of-the-art RNNs, while having less computational time. Finally, the practicality of the APTC-RNN is verified by applying it to the UR3 robotic arm and multiagent systems.

In general, the motion control problem of redundant robot manipulators (RRMs) can be transformed into a constrained time-varying quadratic programming (TVQP) problem. Recently, various recurrent neural networks (RNNs) with predefined time convergence (PTC) abilities have been proposed to solve this constrained TVQP problem in real-time. However, there is still a lack of a unified paradigm to guide researchers and engineers design such RNNs more effectively based on specific requirements. To bridge this gap, we propose a unified paradigm derived from a novel segmentation evolution formula incorporating a special$\mathfrak{B}$–Classfunction. This paradigm enables the construction of various RNNs, collectively referred to as unified arbitrarily predefined-time convergent RNNs (U-APTC-RNNs). Compared with most existing RNNs, the constructed U-APTC-RNN has two significant advantages: 1) it has the arbitrarily PTC (APTC) ability, meaning its actual convergence time can be arbitrarily and precisely predefined without setting other model parameters and 2) using a novel piecewise computation strategy, redundant nonlinear calculations are effectively minimized, leading to a notable reduction in computational costs. The stability and APTC ability of the constructed U-APTC-RNN are demonstrated through detailed theoretical analysis. Numerical simulation experiments confirm the APTC capabilities of various U-APTC-RNNs constructed using the proposed unified paradigm. Comparative experiments show that U-APTC-RNN has more competitive convergence performance and lower computational cost than other state-of-the-art RNNs with PTC abilities. Finally, simulation and physical motion control experiments on the Jaco and UR5 robotic arms demonstrate the superiority and practicality of the proposed U-APTC-RNN.

Players-based emotion recognition can help the understanding game players’ emotional states, contributing to the improvement of the game's quality and value. This article develops a hybrid neural network learning framework called convolutional smooth feedback fuzzy network (CSFFN) to detect a player's emotional states in real-time during a gaming process based on electroencephalogram (EEG) signals. Specifically, CSFFN rationally combines a convolutional neural network (CNN), a fuzzy neural network (FNN), and a recurrent neural network (RNN). CNN not only captures spatial characteristics between EEG signals from different channels but also eliminates noise from EEG signals, improving the accuracy and anti-noise performance in game emotion recognition. FNN extracts the membership degree of a player's different emotional states, further improving the emotion recognition accuracy. Since a player's current emotional state is influenced by the previous emotional states during the game process, RNN is employed to capture the temporal characteristics of EEG signals, better improving the emotion recognition accuracy. Experimental results show that CSFFN has higher recognition accuracy and noise resistance in identifying four emotional states (happiness, sadness, superiority, and anger) compared to support vector machine (SVM) with different kernels, linear discrimination analysis (LDA), AlexNet, and VGG16 methods.

Existing driving fatigue detection methods rarely consider how to effectively fuse the advantages of the electroencephalogram (EEG) and electrocardiogram (ECG) signals to enhance detection performance under noise conditions. To address the issues, this article proposes a new type of the deep learning (DL) framework based on EEG and ECG called the product fuzzy convolutional network (PFCN). It should be noted that this article first investigates how to fuse EEG and ECG signals to deal with driving fatigue detection under noise conditions in both simulated and real-field driving environments. Specifically, the PFCN includes three subnetworks. The first uses a fuzzy neural network (FNN) with feedback and a product layer, effectively capturing the particularity and temporal variation of high-dimensional EEG signals and reducing the time-space complexity. The second subnetwork uses a 1-D convolution to convert the ECG data into feature sequences, providing high accuracy and low computational complexity in ECG data classification. The third subnetwork proposes a fusion-separation mechanism to effectively fuse the extracted ECG and EEG features, suppressing the noise interference and ensuring higher detection accuracy. To evaluate the performance of PFCN, a series of experiments has been set up in both simulated and real-field driving environments. The results indicate that the proposed PFCN model has better robustness and detection accuracy compared with several mainstream fatigue detection models.

Driver fatigue monitoring is very important for driving safety, and many intricate factors while driving make fatigue monitoring harder. To effectively predict driving fatigue, this article proposes a new deep learning framework called TSK-type convolution recurrent fuzzy network (TCRFN) based on the spatial and temporal characteristics of electroencephalogram (EEG) signals. In TCRFN, the convolution block is first introduced to extract spatial dependencies from EEG signals. Furthermore, since EEG noise has a strong spatial dependence, this convolutional neural networks is used to reduce the impact of noise. Additionally, a new local feedback method in fuzzy neural network is proposed to process the EEG signals, which can better capture the temporal dependencies from EEG signals. Finally, a logarithmic spatial firing layer function is used in the proposed TCRFN. The activation performance of this function is smoother, which allows more feature numbers and provides better prediction. The experimental results show that the proposed TCRFN model has better antinoise performance and prediction accuracy compared with other widely used and state-of-the-art models.

Driving fatigue can be detected by measuring drivers' heart rate with a wearable device or extracting their facial features with an RGB camera. However, a wearable device causes inconvenience and discomfort to the driver, and an RGB camera's detection accuracy may be affected by light, glasses, and head orientation. Furthermore, most existing methods ignored the temporal information of fatigue features and the relationship between the features, lowering recognition accuracy. Additionally, some existing fatigue detection methods focused on dealing with fatigue features with a temporal slice, ignoring temporal variations in the features. To address these problems, a single RGB-D camera is first used to extract three fatigue features: heart rate, eye openness level, and mouth openness level. More importantly, this paper proposes a novel multimodal fusion recurrent neural network (MFRNN), integrating the three features to improve the accuracy of driver fatigue detection. Specifically, a recurrent neural network (RNN) layer is applied in the MFRNN to obtain the temporal information of the features. Since the heart rate feature is a physiological signal extracted indirectly, it contains more noise and is fuzzier than the other features. To deal with the fuzziness and noise, we combine fuzzy reasoning with RNN to extract the temporal information of the heart rate. To identify the relationship between the features, we develop a new relationship layer containing a two-level RNN, for which the input is the temporal information of the features. Both the simulation and field experiment results show that the proposed method provides better performance than similar methods.

Traditional human-robot interfaces usually have limitations in accuracy and/or operational space. This article proposes a natural human-robot interface using an adaptive tracking method, which can effectively expand the operational space while ensuring high accuracy. The natural interface allows the robot to directly reproduce the user's hand movement, making the interaction more intuitive and natural. The leap motion is fixed on the Cartesian platform to capture the movement of the user's hand. Because the Cartesian platform follows the hand and keeps the hand in the center of the detection area, the measurement accuracy is improved and the measurement space can be extended. During the process of acquiring gesture data, the measurement errors were found to increase over time because of the inherent noise of the sensor. To deal with this problem, the unscented Kalman filter is applied to estimate the position of the hand. Moreover, an adaptive velocity control method is proposed to improve the operation accuracy and reduce the task execution time with the consideration of users' habits and easiness of usage. The effectiveness of this interface is verified by a series of experiments, and the results show that the proposed interface can be used by nonprofessional users for object operation tasks and can provide users with superior interactive experiences.

In this article, a novel human-machine interface, in which two Leap Motion (LM) controllers and a coil are attached to a Cartesian platform to provide contactless electromagnetic force feedback for enhancing the accuracy and efficiency of human-robot manipulation tasks is presented. To implement such an interface, an interval Kalman filter, an improved particle filter, and a mean filter are integrated to estimate accurately the position and orientation of the hand gesture tracked by the two LM controllers, and to smoothen the movement of the Cartesian platform. The back propagation neural network is employed to regulate the electric currents of the coil attached to the Cartesian platform for accurate force feedback. A series of comparative experiments are performed, and the results show that the presented interface greatly improved the efficiency and accuracy of human-robot manipulation tasks in comparison with existing methods, indicating its great potentials for many industry scenarios.

A novel meshless deformation model of biological soft tissue, which is mainly based on the radial basis function point interpolation, is presented for interactive simulation applications such as virtual surgery simulators. Compared with conventional mesh models, the proposed model is particularly suitable for simulating large deformation, sucking and cutting tasks since there is no need to maintain grid information. Kelvin viscoelasticity, which represents relaxation, creep, and hysteresis of soft tissue, is integrated into the proposed model, making the simulation much more realistic than many existing meshless models. To verify the validity of the proposed model, a biomechanical test was performed on real-life biological tissue and the results show that the maximum relative error between the forces from the biomechanical test and those obtained from the model is less than 5.8%. The proposed model was also implemented on a neurosurgery simulator, which showed that the deformation of the brain tumor can be simulated in a high degree of accuracy with real-time performance. In particular, the error and distortion from the remeshing process inherited in conventional mesh models when deformation is large are avoided.

Unsupervised image segmentation is a fundamental but challenging problem in computer vision. In this paper, we propose a novel unsupervised segmentation algorithm, which could find diverse applications in pattern recognition, particularly in computer vision. The algorithm, named Two-stage Fuzzy c-means Hybrid Approach (TFHA), adaptively clusters image pixels according to their multichannel Gabor responses taken at multiple scales and orientations. In the first stage, the fuzzy c-means (FCM) algorithm is applied for intelligent estimation of centroid number and initialization of cluster centroids, which endows the novel segmentation algorithm with adaptivity. To improve the efficiency of the algorithm, we utilize the Gray Level Co-occurrence Matrix (GLCM) feature extracted at the hyperpixel level instead of the pixel level to estimate centroid number and hyperpixel-cluster memberships, which are used as initialization parameters of the following main clustering stage to reduce the computational cost while keeping the segmentation performance in terms of accuracy close to original one. Then, in the second stage, the FCM algorithm is utilized again at the pixel level to improve the compactness of the clusters forming final homogeneous regions. To examine the performance of the proposed algorithm, extensive experiments were conducted and experimental results show that the proposed algorithm has a very effective segmentation results and computational behavior, decreases the execution time and increases the quality of segmentation results, compared with the state-of-the-art segmentation methods recently proposed in the literature.