In recent years, cross-frequency coupling (CFC) has emerged as a valuable tool in the study of a wide range of cognitive processes due to the strong evidence of its functional role in neural computation and communication. CFC computed from electroencephalography (EEG) signals provides powerful information for detecting certain neurological conditions associated with atypical cerebral lateralization. The use of deep learning (DL) in this context offers several advantages, including improved scalability and adaptability to individual variability. However, it presents several significant challenges related to the limited availability of labelled samples and the high-dimensional and noisy nature of EEG data, which can lead to overfitting, poor generalization, and temporal and spatial variability between subjects. In this work, we propose a novel deep learning approach to reveal lateralization patterns based on inter-hemispheric functional differences via CFC. To overcome the challenges associated to the use of DL in this context, we propose the use of synthetic signals for pre-training the neural network that computes a specific type of CFC, phase-amplitude coupling (PAC), and a symmetric architecture for evaluating inter-hemispheric differences. Finally, our model incorporates a custom attention layer designed to learn the most relevant information across different EEG channels and its relative importance, further enhancing its ability to detect subtle hemispheric differences and providing the necessary explainability for clinical applications. The results demonstrate a good classification performance (AUC up to 0.85) in assessing lateralization, providing explainable insights into the mechanisms of the disorder. This may aid in early detection and provide a better understanding of the neural basis associated with this condition.

With the rapid development of deep generative models (DGMs), the performance of decoding language and reconstructing images from Functional Magnetic Resonance Imaging (fMRI) has been improved. Nevertheless, the accurate representation of brain activity remains highly challenging, primarily due to the limited paired samples and the low signal-to-noise ratios of fMRI. To tackle these challenges, we introduce Cogformer, a unified multi-scale brain representation method. It is the first to learn brain representation from multi-scale fMRI activities via self-attention, and integrate a synchronized decoding and dynamic decoupling strategy for structural and semantic features through cross-attention. We conduct a systematic evaluation of Cogformer on the large-scale Natural Scenes Dataset (NSD) across a broad range of visual decoding tasks, including category classification, multi-label classification, image retrieval, image captioning, and image reconstruction. To the best of our knowledge, this represents the most extensive task coverage reported in related research. Cogformer achieves superior performance compared to a range of transformer-based baselines in category classification, multi-label classification, and image retrieval tasks. Moreover, in the more challenging tasks of image captioning and image reconstruction, Cogformer leverages a prior diffusion module to enhance the alignment with image semantics. This further improves the semantic consistency for caption generation and visual fidelity in image reconstruction. Across multiple evaluation metrics, Cogformer demonstrates competitive performance against existing state-of-the-art (SOTA) methods, highlighting its strong decoding capabilities and generalization potential.

Research on electroencephalogram (EEG)-based person identification is increasing because EEG signals must be collected from the living body, making them difficult to steal or alter. However, EEG signals are greatly influenced by subjects’ states, and most studies on EEG-based person identification have overlooked this influence. In this study, we proposed an interpretable contrastive learning transformer to tackle the impact of state changes on EEG-based person identification. Contrastive learning transformers construct pairs of EEG signal feature samples to capture state-independent and identity-distinct features. Specifically, the power spectral density (PSD) of EEG signals from the same user in different paradigms is used as positive samples, while the PSD from other users is used as negative samples. Pairs of samples are encoded to obtain corresponding features and then projected into a contrastive space through a multi-layer perceptron. Then, the NT-Xent loss function minimizes the distance between positive samples within the same batch and maximizes the distance between negative samples. Finally, to eliminate bias between positive sample pairs from different paradigms, we introduced the cross-paradigm alignment loss for the first time to capture individual consistency. We evaluated our model on two datasets. Dataset 1 contains EEG signals from 109 individuals, recorded across multiple paradigms designed to elicit different states. Dataset 2 consists of EEG signals from 71 individuals, collected across two sessions, with each session including two paradigms. We evaluated the accuracy of both single-paradigm and cross-paradigm recognition. Our proposed model outperforms state-of-the-art models for EEG-based person identification. We also conducted experiments on electrode attention visualization to capture the brain regions that the model focuses on, and the results demonstrate that, unlike in a single-paradigm, models trained in cross-paradigm focus on fewer electrodes and more concentrated regions.

For privacy protection of subjects in electroencephalogram (EEG)-based brain-computer interfaces (BCIs), using source-free domain adaptation (SFDA) for cross-subject recognition has proven to be highly effective. However, updating and storing a model trained on source subjects for each new subject can be inconvenient. This paper extends Euclidean alignment (EA) to propose adaptive Euclidean alignment (AEA), which learns a projection matrix to align the distribution of the target subject with the source subjects, thus eliminating domain drift issues and improving model classification performance of subject-independent BCIs. Combining the proposed AEA with various existing SFDA methods, such as SHOT, GSFDA, and NRC, this paper presents three new methods: AEA-SHOT, AEA-GSFDA, and AEA-NRC. In our experimental studies, these AEA-based SFDA methods were applied to four well-known deep learning models (i.e., EEGNet, Shallow ConvNet, Deep ConvNet, and MSFBCNN) on two motor imagery (MI) datasets, one event-related potential (ERP) dataset and one steady-state visual evoked potentials (SSVEP) dataset. The advanced cross-subject EEG classification performance demonstrates the efficacy of our proposed methods. For example, AEA-SHOT achieved the best average accuracy of 81.4% on the PhysioNet dataset.

Dissolved Gas Analysis (DGA) is the most commonly used method for power transformer fault diagnosis. However, very few reliable and labeled fault DGA samples are available in the transformer substation whilst DGA data without labels is easier to obtain, which makes it difficult to train fault detectors in high-dimensional input space or select features using wrapper methods. Therefore, in order to improve the fault diagnosis accuracy using limited labeled DGA samples but more unlabeled DGA data, this paper proposes a novel multi-filter semi-supervised feature selection method for selecting optimal DGA features and building effective fault diagnosis models. A confidence criterion is also proposed for selecting high confidence unlabeled data to expand the training data set. Five filter techniques based on different evaluation criteria are employed to rank input DGA features, and a feature combination method is then applied to aggregate feature ranks by multiple filters and form a lower-dimensional candidate feature subset. The proposed method has been tested by using the IEC T10 dataset and compared with traditional supervised diagnostic models. The results show that the proposed method works well in optimizing DGA features and improving fault diagnosis accuracy significantly. Besides, the robustness of the selection of optimal feature subset is validated by testing DGA samples from the local power utility.

Abstract Combining convolutional neural networks (CNNs) and recurrent neural networks (RNNs) produces a powerful architecture for video classification problems as spatial–temporal information can be processed simultaneously and effectively. Using transfer learning, this paper presents a comparative study to investigate how temporal information can be utilized to improve the performance of video classification when CNNs and RNNs are combined in various architectures. To enhance the performance of the identified architecture for effective combination of CNN and RNN, a novel action template-based keyframe extraction method is proposed by identifying the informative region of each frame and selecting keyframes based on the similarity between those regions. Extensive experiments on KTH and UCF-101 datasets with ConvLSTM-based video classifiers have been conducted. Experimental results are evaluated using one-way analysis of variance, which reveals the effectiveness of the proposed keyframe extraction method in the sense that it can significantly improve video classification accuracy.

Facial expression recognition (FER) is an important research field for human-machine interfaces, and FER by deep learning from small training datasets is still challenging. This paper proposes a new generative adversarial network (GAN) model to transfer neutral face images to images with diverse facial expressions to deal with the problem of expressional data scarcity in deep learning for FER based on a small set of training samples. To mitigate distortions and overfitting that often happen in training GANs with small training datasets, a novel GAN architecture is proposed, which consists of a generator with two encoders and two decoders, two discriminators, and a feature extractor. Specifically, a feature map mechanism is proposed to discover regional feature differences between images in the source domain and target domain, which makes the proposed GAN architecture be able to not only generate desirable facial expression images but also maintain the original characters in input neutral face images. Experimental results show that, by using the proposed GAN to augment a training dataset of images with up to 7 facial expressions, the FER accuracy of several deep neural networks tested in the experiments can be significantly improved by over 10%.

It is difficult to achieve high performance without sufficient training data for deep convolutional neural networks (DCNNs) to learn. Data augmentation plays an important role in improving robustness and preventing overfitting in machine learning for many applications such as image classification. In this paper, a novel method for data augmentation is proposed to solve the problem of machine learning with small training datasets. The proposed method can synthesise similar images with rich diversity from only a single original training sample to increase the number of training data by using generative adversarial networks (GANs). It is expected that the synthesised images possess class-informative features, which may be in the validation or testing data but not in the training data due to that the training dataset is small, and thus they can be effective as augmented training data to improve the classification accuracy of DCNNs. The experimental results have demonstrated that the proposed method with a novel GAN framework for image training data augmentation can significantly enhance the classification performance of DCNNs for applications where original training data is limited.

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record.This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article.Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

In this paper, a new co-training algorithm based on modified Fisher's Linear Discriminant Analysis (FLDA) is proposed for semi-supervised learning, which only needs a small set of labeled samples to train classifiers and is thus very useful in applic

Among the many paradigms used in brain-computer interface (BCI), steady state visual evoked potential (SSVEP) offers the quickest response; however it is disadvantageous from the point of view of visual fatigue, which prevents subjects from prolonged usage of visual stimuli especially when LEDs are used. In this paper, we propose a visual stimulator using readily available RGB LEDs with clear and frosted glass, with the latter being tested for performance and qualitative user comfort using electroencephalogram (EEG) data from four subjects. Furthermore, we also compare frosted and clear stimuli for three colours Red, Green and Blue with frequency values of 7, 8, 9 and 10 Hz. The results using band-pass filtering and Fast Fourier Transform showed that 7 Hz Green clear LED stimuli gave the highest response in general, although all the subjects indicated that they were more comfortable with frosted LED stimuli.

A novel presentation for channel selection problem in Brain-Computer Interfaces (BCI) is introduced here. Continuous presentation in a projected two-dimensional space of the Electroencephalograph (EEG) cap is proposed. A multi-objective particle swarm optimization method (D2MOPSO) is employed where particles move in the EEG cap space to locate the optimum set of solutions that minimize the number of selected channels and the classification error rate. This representation focuses on the local relationships among EEG channels as the physical location of the channels is explicitly represented in the search space avoiding picking up channels that are known to be uncorrelated with the mental task. In addition continuous presentation is a more natural way for problem solving in PSO framework. The method is validated on 10 subjects performing right-vs-left motor imagery BCI. The results are compared to these obtained using Sequential Floating Forward Search (SFFS) and shows significant enhancement in classification accuracy but most importantly in the distribution of the selected channels.

This paper presents a comparative study among three evolutionary and search based methods to solve the problem of channel selection for Brain-Computer Interface (BCI) systems. Multi-Objective Particle Swarm Optimization (MOPSO) method is compared to Multi-Objective Evolutionary Algorithm based on Decomposition (MOEA/D) and single objective Sequential Floating Forward Search (SFFS) method. The methods are tested on the first data set for BCI-Competition IV. The results show the usefulness of the multi-objective evolutionary methods in achieving accuracy results similar to the extensive search method with fewer channels and less computational time.

In this paper, a method for constructing Takagi-Sugeno (TS) fuzzy system from data is proposed with the objective of preserving TS submodel comprehensibility, in which linguistic modifiers are suggested to characterize the fuzzy sets. A good property held by the proposed linguistic modifiers is that they can broaden the cores of fuzzy sets while contracting the overlaps of adjoining membership functions (MFs) during identification of fuzzy systems from data. As a result, the TS submodels identified tend to dominate the system behaviors by automatically matching the global model (GM) in corresponding subareas, which leads to good TS model interpretability while producing distinguishable input space partitioning. However, the GM accuracy and model interpretability are two conflicting modeling objectives, improving interpretability of fuzzy models generally degrades the GM performance of fuzzy models, and vice versa. Hence, one challenging problem is how to construct a TS fuzzy model with not only good global performance but also good submodel interpretability. In order to achieve a good tradeoff between GM performance and submodel interpretability, a regularization learning algorithm is presented in which the GM objective function is combined with a local model objective function defined in terms of an extended index of fuzziness of identified MFs. Moreover, a parsimonious rule base is obtained by adopting a QR decomposition method to select the important fuzzy rules and reduce the redundant ones. Experimental studies have shown that the TS models identified by the suggested method possess good submodel interpretability and satisfactory GM performance with parsimonious rule bases.

In this paper, a new scheme for constructing parsimonious fuzzy classifiers is proposed based on the L2-support vector machine (L2-SVM) technique with model selection and feature ranking performed simultaneously in an integrated manner, in which fuzzy rules are optimally generated from data by L2-SVM learning. In order to identify the most influential fuzzy rules induced from the SVM learning, two novel indexes for fuzzy rule ranking are proposed and named as alpha-values and omega-values of fuzzy rules in this paper. The alpha-values are defined as the Lagrangian multipliers of the L2-SVM and adopted to evaluate the output contribution of fuzzy rules, while the omega-values are developed by considering both the rule base structure and the output contribution of fuzzy rules. As a prototype-based classifier, the L2-SVM-based fuzzy classifier evades the curse of dimensionality in high-dimensional space in the sense that the number of support vectors, which equals the number of induced fuzzy rules, is not related to the dimensionality. Experimental results on high-dimensional benchmark problems have shown that by using the proposed scheme the most influential fuzzy rules can be effectively induced and selected, and at the same time feature ranking results can also be obtained to construct parsimonious fuzzy classifiers with better generalization performance than the well-known algorithms in literature.

Artificial neural networks have been traditionally employed to learn and compute the inverse kinematics of a robotic arm. However, the inverse kinematics model of a typical robotic arm with joint limits is a multi-valued and discontinuous function. Because it is difficult for a multilayer neural network to approximate this type of function, an accurate inverse kinematics model cannot be obtained by using a single neural network. In order to overcome the difficulties of inverse kinematics learning, we propose a novel modular neural network system that consists of a number of expert modules, where each expert approximates a continuous part of the inverse kinematics function. The proposed system selects one appropriate expert whose output minimizes the expected position/orientation error of the end-effector of the arm. The system can learn a precise inverse kinematics model of a robotic arm with equal or more degrees of freedom than that of its end-effector. However, there are robotic arms with fewer degrees of freedom, where the system cannot learn their precise inverse kinematics model. We have adopted a modified Gauss-Newton method for finding the least-squares solution to address this issue. Through the modifications presented in this paper, the improved modular neural network system can obtain a precise inverse kinematics model of a general robotic arm.

A fuzzy set merging (FSM) algorithm is proposed in order to generate distinguishable fuzzy sets. A relative compactness measure is defined to characterize the homogenous information that one pattern shares with its neighbors, and a so-called "local" entropy is employed to evaluate the distinguishability of fuzzy sets. By maximizing this entropy measure the optimal number of merged fuzzy sets with good distinguishability can be obtained, which preserve the information of original fuzzy sets as much as possible. Furthermore, we propose a scheme to optimize the input space partitioning for a Takagi-Sugeno (TS) fuzzy model by using the FSM algorithm. As a result, a good trade-off between global approximation ability and interpretability in input space partitioning is achieved in the TS model.

We present a new Tagaki-Sugeno (TS) type model whose membership functions (MFs) are characterized by linguistic modifiers. As a result, during adaptation, the trained local models tend to become the tangents of the global model, leading to good model interpretability. In order to prevent the global approximation ability from being degraded, an index of fuzziness is proposed to evaluate linguistic modification for MFs with adjustable crossover points. A new learning scheme is also developed, which uses the combination of global approximation error and the fuzziness index as its objective function. By minimizing the multiple objective performance measure, a tradeoff between the global approximation and local model interpretation can be achieved. Experimental results show that by the proposed method, good interpretation of local models and transparency of input space partitioning can be obtained for the TS model, while at the same time the global approximation ability is still preserved.

In this paper, a novel procedure for normalising Mercer kernel is suggested firstly. Then, the normalised Mercer kernel techniques are applied to the fuzzy c-means (FCM) algorithm, which leads to a normalised kernel based FCM (NKFCM) clustering algorithm. In the NKFCM algorithm, implicit assumptions about the shapes of clusters in the FCM algorithm is removed so that the new algorithm possesses strong adaptability to cluster structures within data samples. Moreover, a new method for calculating the prototypes of clusters in input space is also proposed, which is essential for data clustering applications. Experimental results on several benchmark datasets have demonstrated the promising performance of the NKFCM algorithm in different scenarios.

Fuzzy local linearization (FLL) is a useful divide-and-conquer method for coping with complex problems such as modeling unknown nonlinear systems from data for state estimation and control. Based on a probabilistic interpretation of FLL, the paper proposes a hybrid learning scheme for FLL modeling, which uses a modified adaptive spline modeling (MASMOD) algorithm to construct the antecedent parts (membership functions) in the FLL model, and an expectation-maximization (EM) algorithm to parameterize the consequent parts (local linear models). The hybrid method not only has an approximation ability as good as most neuro-fuzzy network models, but also produces a parsimonious network structure (gain from MASMOD) and provides covariance information about the model error (gain from EM) which is valuable in applications such as state estimation and control. Numerical examples on nonlinear time-series analysis and nonlinear trajectory estimation using FLL models are presented to validate the derived algorithm.

Fuzzy local linearization is compared with local basis function expansion for modeling unknown nonlinear processes. First-order Takagi-Sugeno fuzzy model and the analysis of variance (ANOVA) decomposition are combined for the fuzzy local linearization of nonlinear systems, in which B-splines are used as membership functions of the fuzzy sets for input space partition. A modified algorithm for adaptive spline modeling of observation data (MASMOD) is developed for determining the number of necessary B-splines and their knot positions to achieve parsimonious models. This paper illustrates that fuzzy local linearization models have several advantages over local basis function expansion based models in nonlinear system modeling.

Model-based methods for the state estimation and control of linear systems have been well developed and widely applied. In practice, the underlying systems are often unknown and nonlinear. Therefore, data based model identification and associated linearization techniques are very important. Local linearization and feedback linearization have drawn considerable attention in recent years. In this paper, linearization techniques using neural networks are reviewed, together with theoretical difficulties associated with the application of feedback linearization. A recurrent neurofuzzy network with an analysis of variance (ANOVA) decomposition structure and its learning algorithm are proposed for linearizing unknown discrete-time nonlinear dynamic systems. It can be viewed as a method for approximate feedback linearization, as such it enlarges the class of nonlinear systems that can be feedback linearized using neural networks. Applications of this new method to state estimation are investigated with realistic simulation examples, which shows that the new method has useful practical properties such as model parametric parsimony and learning convergence, and is effective in dealing with complex unknown nonlinear systems.

The paper describes a high performance offline system for recognizing hand printed numerals. An innovative verification module is applied which drastically improves the recognition rate. The approaches used in the modules are described. The importance of the verification module is analysed in detail. A practical automatic form reading system TOCR V1.0 was developed based on the algorithms. The system was put into practical use in several provinces of China for statistical analysis of Revenue China. Test results are given based on: 1) data collected when the system was used in China, as well as 2) the CENPARMI database.