Failure prediction is crucial for ensuring the stability of cloud computing systems and has garnered extensive attention from both academia and industry. Generally, data used for prediction includes two modalities: 1) Text data, such as logs; and, 2) Numerical data, such as error counts and monitoring metrics. However, most existing failure prediction algorithms for cloud computing only focus on a single modality. The lack of high-quality multimodal datasets from real-world production environments constrains academic research on multimodal failure prediction. To fill this gap, this paper releases a large multimodal dataset of operation data from the Alibaba cloud computing platform, namely, Logs and Metrics Integration Dataset (LMID). It consists of 100 million pieces of logs (textual data) and 37 dimensions of monitoring metrics (numerical data) from 220,000 physical machines. To our knowledge, it is the first multimodal dataset for cloud computing system failure prediction, and is expected to greatly benefit the community. This paper provides a detailed introduction to the construction of LMID, its contents, and the performance of state-of-the-art algorithms on it. It also conducts extensive experiments to reveal a new insight that cross-modality connections are effective for failure prediction. LMID is now available at https://huggingface.co/datasets/AliyunECSAlgos/LMID.

Embodied intelligence and related disciplines have identified several mechanisms that help embodied agents learn how to solve complex problems. Reinforcement learning (RL) is one of the most promising computational approaches toward enhancement of the learning-based problem-solving abilities of such agents. Given the recent rapid evolution of artificial intelligence, RL has become a keystone technology, accelerating scientific discoveries and also finding applications in many other domains. In RL, an agent collects data when interacting with the environment, which optimizes a policy ensuring a higher return. Further improvement requires more exploration of the action space. However, not all actions in that space are safe and acceptable. The exploration of an agent must be constrained. In this work, a novel mirror descent safe policy optimization (MDSPO) algorithm is proposed to ensure the safety of an RL agent. The algorithm leverages mirror descent optimization to maximize the return while satisfying the safety constraint. A novel optimization objective is formulated, and an innovative three-stage optimization strategy is employed-comprising gradient descent without the cost constraint, projection onto the nonparametric policy space with the cost constraint, and projection onto the parametric policy space. Compared to previous methods, MDSPO is a simple and easy to implement first-order approach, which does not impose a hard constraint on the trust region. Theoretical analysis of the MDSPO reveals a lower bound on return improvement and an upper bound on constraint violation at the time of each policy update. The numerical results obtained from two sets of different constrained locomotive experiments demonstrate that MDSPO improves the average return by about 12% and better satisfies the cost constraints than other state-of-the-art methods do. In a real-world obstacle avoidance experiment using an unmanned surface vessel, MDSPO both finds the optimal path and guarantees agent safety.

Emotion recognition is an important component of affective computing, and also human-machine interaction. Unimodal emotion recognition is convenient, but the accuracy may not be high enough; on the contrary, multi-modal emotion recognition may be more accurate, but it also increases the complexity and cost of the data collection system. This paper considers cross-modal emotion recognition, i.e., using both electroencephalography (EEG) and eye movement in training, but only EEG or eye movement in test. We propose cross-modal contrastive representation distillation (CMCRD), which uses a pre-trained eye movement classification model to assist the training of an EEG classification model, improving feature extraction from EEG signals, or vice versa. During test, only EEG signals (or eye movement signals) are acquired, eliminating the need for multi-modal data. CMCRD not only improves the emotion recognition accuracy, but also makes the system more simplified and practical. Experiments using three different neural network architectures on three multi-modal emotion recognition datasets demonstrated the effectiveness of CMCRD. Compared with the EEG-only model, it improved the average classification accuracy by about 6.2%.

User identification technologies are essential for ensuring security and privacy. Compared to conventional biometric identification methods, electroencephalogram (EEG)-based brainprint recognition provides unique advantages, including non-replicability, resistance to coercion, and inherent liveness detection. However, existing EEG-based brainprint recognition methods are typically tailored for specific tasks and evaluated under conditions that differ substantially from real-world use. To overcome these limitations, we propose BrainprintNet, a convolutional neural network architecture integrating fine-grained filter banks, grouped multiscale temporal convolutions, and cross-band spatial fusion to enhance EEG-based brainprint recognition. BrainprintNet surpasses previous architectures in challenging scenarios involving simultaneous cross-session and cross-task recognition, demonstrating its generalization ability under strict simulation for real-world applications. Comprehensive experiments were conducted using three publicly available datasets encompassing nine distinct tasks. Furthermore, visualization of the learned network weights revealed strong correlations between user identity and specific EEG frequency subbands and channels. The proposed BrainprintNet significantly advances the accuracy, flexibility, and practical applicability of EEG-based brainprint recognition systems.

Electroencephalography (EEG)-based brain-computer interfaces (BCIs) transform spontaneous/evoked neural activity into control commands for external communication. While convolutional neural networks (CNNs) remain the mainstream backbone for EEG decoding, their inherently short receptive field makes it difficult to capture long-range temporal dependencies and global inter-channel relationships. Recent CNN-Transformer (Conformer) hybrids partially address this issue, but most adopt a serial design, resulting in suboptimal integration of local and global features, and often overlook explicit channel-wise modeling. To address these limitations, we propose DBConformer, a dual-branch convolutional Transformer network tailored for EEG decoding. It integrates a temporal Conformer to model long-range temporal dependencies and a spatial Conformer to extract inter-channel interactions, capturing both temporal dynamics and spatial patterns in EEG signals. A lightweight channel attention module further refines spatial representations by assigning data-driven importance to EEG channels. Extensive experiments under four evaluation settings on three paradigms, including motor imagery, seizure detection, and steady-state visual evoked potential, demonstrated that DBConformer consistently outperformed 13 competitive baseline models, with over an eight-fold reduction in parameters than current high-capacity EEG Conformer architecture. Furthermore, the visualization results confirmed that the features extracted by DBConformer are physiologically interpretable and aligned with prior knowledge. The superior performance and interpretability of DBConformer make it reliable for accurate, robust, and explainable EEG decoding.

As software complexity rises, research on vulnerability detection becomes increasingly important. Deep learning-based vulnerability detection, an emerging approach, can segment code and identify hidden vulnerability patterns. However, challenges remain: (1) accurately correlating code slices with scripts to minimize false positives; (2) enhancing the precision of root cause localization for vulnerable scripts. To alleviate these, this paper introduces a vulnerability detection and root cause localization approach leveraging large language models (LLMs). The approach preprocesses C/C++ source code, extracts graph structures, and combines them with the script to form prompts. A novel Hierarchical Regulation for Parameter-Efficient Language Model Tuning (HR-PELT) approach fine-tunes the LLM for vulnerability detection by maintaining parameters of shallow layers substantially preserved while enhancing the adaptability of deep layers. For root cause localization, we similarly create prompts and fine-tune another LLM. Experimental results on three datasets demonstrated improvements: in vulnerability detection, our approach boosted average Accuracy (ACC) by 4.82% and Macro-F1 (M-F1) by 5.29% over the state-of-the-art (SOTA); in root cause localization, it enhanced ACC10%by 4.80% and ACC20%by 3.93%.

Black-Box Knowledge Distillation (B2KD) is a conservative task in cloud-to-edge model compression, emphasizing the protection of data privacy and model copyrights on both the cloud and edge. With invisible data and models hosted on the server, B2KD aims to utilize only the API queries of the teacher model's inference results in the cloud to effectively distill a lightweight student model deployed on edge devices. B2KD faces challenges such as limited Internet exchange and edge-cloud disparity in data distribution. To address these issues, we theoretically provide a new optimization direction from logits to cell boundary, different from direct logits alignment, and formalize a workflow comprising deprivatization, distillation, and adaptation at test time. Guided by this, we propose a method, Mapping-Emulation KD (MEKD), to enhance the robust prediction and anti-interference capabilities of the student model on edge devices for any unknown data distribution in real-world scenarios. Our method does not differentiate between treating soft or hard responses and consists of: 1) deprivatization: emulating the inverse mapping of the teacher function with a generator, 2) distillation: aligning low-dimensional logits of the teacher and student models by reducing the distance of high-dimensional image points, and 3) adaptation: correcting the student's online prediction bias through a graph propagation-based only-forward test-time adaptation algorithm. Our method demonstrates inspiring performance for edge model distillation and adaptation across different teacher-student pairs. We validate the effectiveness of our method on multiple image recognition benchmarks and various Deep Neural Network models, achieving state-of-the-art performance and showcasing its practical value in remote sensing image recognition applications.

Physiological signal-based emotion recognition requires effective fusion of multi-modal physiological signals to improve recognition accuracy. In this paper, a multi-kernel embedding fusion framework (MKEFF) is proposed for multi-modal physiological signal emotion recognition. Specifically, multi-kernel learning and kernel approximation techniques are used to compute the multi-kernel embeddings of the original feature vectors of each modality independently. The embeddings are then fed in parallel to their respective representation learning layer, where the proposed sparse relation learning method is applied to all the modalities to explore the correlation and diversity among them. Finally, a distribution alignment based fusion method is proposed to align each modality in the subspace, and a weighted summation fusion is performed to obtain the fused representations. Extensive cross-subject emotion recognition experiments are conducted on three public datasets, DEAP, DECAF, and SEED-IV, to evaluate the proposed method. The experimental results demonstrate that the proposed method achieves better classification performance and interpretability than the state-of-the-art methods.

Electroencephalogram (EEG) based sleep stage classification is very important in sleep quality analysis and the treatment of sleep disorders. Deep learning based automated sleep staging has achieved promising performance. However, it has not been widely adopted in clinical practice, due to the domain shift problem and insufficient labeled training data, especially for patients. To cope with these problems, this paper proposes a Transformer-based semi-supervised domain adaptation (SSDA) approach for EEG-based sleep stage classification. It uses two class tokens to extract knowledge from the source and target domains separately. Then, a training-test strategy with adaptive entropy-weighted ensemble, attention-based adaptation and consistency regularization is used to improve the target domain performance. Experiments on two public datasets demonstrated the effectiveness of the proposed approach.

The cross-task gap presents a significant challenge for multimodal models because of the differences in input-output workflows. For instance, multimodal pre-trained transformers may encounter uni-modal data during testing. To mitigate the gap, this paper introduces a Transformer in Multimodal Sentiment Analysis under Missing Modalities (TMMM) aims to perform well using missing-modal data during testing. TMMM uses a missing multimodal training approach to prevent accuracy degradation in testing. At the same time, a new network architecture allows the model to reconstruct missing modalities during testing. Classification token fusion and Mixture-of-Experts structures further enhance the model’s performance. A pre-training method utilizing contrastive learning, which can construct negative samples with positive samples, is proposed to overcome insufficient labeled data. Our experiments demonstrated the effectiveness of TMMM on two datasets with no modalities missing, i.e., it consistently achieved the highest classification accuracy and Macro-F1, which outperformed the best state-of-the-art baseline on each dataset by about 2% and 2.5%. Additionally, TMMM usually performs better than other baselines on datasets with missing modalities during testing.

Log-based failure prediction helps identify and mitigate system failures ahead of time, increasing the reliability of cloud elastic computing systems.However, most existing log-based failure prediction approaches only focus on semantic information, and do not make full use of the information contained in the timestamps of log messages.This paper proposes time-aware attention-based transformer (TAAT), a failure prediction approach that extracts semantic and temporal information simultaneously from log messages and their timestamps.TAAT first tokenizes raw log messages into specific exceptions, and then performs: 1) exception sequence embedding that reorganizes the exceptions of each node as an ordered sequence and converts them to vectors; 2) time relation estimation that computes time relation matrices from the timestamps; and, 3) time-aware attention that computes semantic correlation matrices from the exception sequences and then combines them with time relation matrices.Experiments on Alibaba Cloud demonstrated that TAAT achieves an approximately 10% performance improvement compared with the state-of-the-art approaches.TAAT is now used in the daily operation of Alibaba Cloud.Moreover, this paper also releases the real-world cloud computing failure prediction dataset used in our study, which consists of about 2.7 billion syslogs from about 300,000 node controllers during a 4-month period.To our * Both authors contributed equally to this research.

A brain-computer interface (BCI) enables direct communication between the brain and an external device. Electroencephalogram (EEG) is the preferred input signal in non-invasive BCIs, due to its convenience and low cost. EEG-based BCIs have been successfully used in many applications, such as neurological rehabilitation, text input, games, and so on. However, EEG signals inherently carry rich personal information, necessitating privacy protection. This paper demonstrates that multiple types of private information (user identity, gender, and BCI-experience) can be easily inferred from EEG data, imposing a serious privacy threat to BCIs, To address this issue, we design perturbations to convert the original EEG data into privacy-protected EEG data, which conceal the private information while maintaining the primary BCI task performance. Experimental results demonstrated that the privacy-protected EEG data can significantly reduce the classification accuracy of user identity, gender and BCI-experience, but almost do not affect at all the classification accuracy of the primary BCI task, enabling user privacy protection in EEG-based BCIs.

Electroencephalogram (EEG)-based seizure sub-type classification enhances clinical diagnosis efficiency. Source-free semi-supervised domain adaptation (SF-SSDA), which transfers a pre-trained model to a new dataset with no source data and limited labeled target data, can be used for privacy-preserving seizure subtype classification. This paper considers two challenges in SF-SSDA for EEG-based seizure subtype classification: 1) How to effectively fuse both raw EEG data and expert knowledge in classifier design? 2) How to align the source and target domain distributions for SF-SSDA? We propose a Knowledge-Data Fusion based SF-SSDA approach, KDF-MutuaISHOT, for EEG-based seizure subtype classification. In source model training, KDF uses Jensen-Shannon Diver-gence to facilitate mutual learning between a feature-driven Decision Tree-based model and a data-driven Transformer-based model. To adapt KDF to a new target dataset, an SF-SSDA algorithm, MutualSHOT, is developed, which features a consistency-based pseudo-label selection strategy. Experiments on the public TUSZ and CHSZ datasets demonstrated that KDF-MutualSHOT outperformed other supervised and source-free domain adaptation approaches in cross-subject seizure subtype classification.

Spiking neural networks (SNNs) aim to simulate real neural networks in the human brain with biologically plausible neurons. The leaky integrate-and-fire (LIF) neuron is one of the most widely studied SNN architectures . However, it has the vanishing gradient problem when trained with backpropagation . Additionally, its neuronal parameters are often manually specified and fixed, in contrast to the heterogeneity of real neurons in the human brain. This paper proposes a gated parametric neuron (GPN) to process spatio-temporal information effectively with the gating mechanism. Compared with the LIF neuron, the GPN has two distinguishing advantages: (1) it copes well with the vanishing gradients by improving the flow of gradient propagation; and, (2) it learns spatio-temporal heterogeneous neuronal parameters automatically. Additionally, we use the same gate structure to eliminate initial neuronal parameter selection and design a hybrid recurrent neural network-SNN structure. Experiments on two spike-based audio datasets demonstrated that the GPN network outperformed several state-of-the-art SNNs, could mitigate vanishing gradients, and had spatio-temporal heterogeneous parameters. Our work shows the ability of SNNs to handle long-term dependencies and achieve high performance simultaneously.

Emotion recognition is a critical component of affective computing. Training accurate machine learning models for emotion recognition typically requires a large amount of labeled data. Due to the subtleness and complexity of emotions, multiple evaluators are usually needed for each affective sample to obtain its ground-truth label, which is expensive. To save the labeling cost, this paper proposes an inconsistency-based active learning approach for cross-task transfer between emotion classification and estimation. Affective norms are utilized as prior knowledge to connect the label spaces of categorical and dimensional emotions. Then, the prediction inconsistency on the two tasks for the unlabeled samples is used to guide sample selection in active learning for the target task. Experiments on within-corpus and cross-corpus transfers demonstrated that cross-task inconsistency could be a very valuable metric in active learning. To our knowledge, this is the first work that utilizes prior knowledge on affective norms and data in a different task to facilitate active learning for a new task, even the two tasks are from different datasets.

Brain-computer interface (BCI) is a direct communication pathway between the brain and an external device. Because of individual differences and non-stationarity of brain signals, a BCI usually needs subject-specific calibration, which is time-consuming and user unfriendly. Sophisticated machine learning approaches can help reduce or even completely eliminate calibrations, improving the utility of BCIs. Recent studies also found that machine learning models in BCIs are vulnerable to adversarial attacks, and brain signals also contain lots of private information, so the security and privacy of BCIs are also important considerations in their commercial applications. This talk will introduce transfer learning approaches for expedite BCI calibration, and their adversarial attack and privacy protection approaches. The ultimate goal is to implement accurate, secure and privacy-preserving BCIs.

On behalf of the organizing committee, we are most delighted to welcome you to join us at 23rd IEEE/ACIS International Conference on Computer and Information Science (ICIS 2023), to be held on June 23-24, 2023 in Wuxi, China. The conference is sponsored by IEEE Computer Society and International Association for Computer and Information Science (ACIS), and in cooperation with Jiangnan University, China.

The modern cloud computing system has evolved into a highly dynamic and complex ecosystem with thousands of modules. Modifications and updates to these modules occur every day to accommodate customer needs. Unfortunately, these frequent changes may introduce anomalies to the system, whose diffusion can undermine the system performance and even cause system outage. Though very important, it is challenging to detect anomalies at the early stages and locate their root causes, due to the complexity of the cloud ecosystem and the huge number of attribute combinations. This paper proposes DARC for high-dimensional diffusing anomaly detection and root cause location in cloud computing systems. DARC uses first two-stage percentile analysis and Mann-Kendall score thresholding to detect rare anomalies, and then a bottom-up search strategy with three computational complexity reduction techniques to efficiently locate the root causes. Extensive experiments showed that DARC is able to accurately and efficiently locate root causes of diffusing anomalies. It has been successfully used in the daily practice of Alibaba Cloud, one of the world’s largest cloud computing service providers.

Electroencephalogram (EEG) based seizure subtype classification plays an important role in clinical diagnostics. However, existing deep learning approaches face two challenges in such applications: 1) convolutional or recurrent neural network based models have difficulty learning long-term dependencies; and, 2) there are not enough labeled seizure sub-type data for training such models. This paper proposes a Transformer-based self-supervised learning model for EEG-based seizure subtype classification, which copes well with these two challenges. Filter bank analysis is first employed to improve Vision Transformer as a Wavelet Transformer (WaT) encoder, which generates multi-grained feature representations of EEG signals. Then, self-supervised learning is used to pre-train WaT from unlabeled EEG data. Experiments on two public datasets demonstrated that Wavelet2Vec outperformed several other supervised and self-supervised models in cross-subject seizure subtype classification.

Continual learning aims to learn on a sequence of new tasks while maintaining the performance on previous tasks. Source-free domain adaptation (SFDA), which adapts a pretrained source model to a target domain, is useful in protecting the source domain data privacy. Generalized SFDA (G-SFDA) combines continual learning and SFDA to achieve outstanding performance on both the source and the target domains. This paper proposes semi-supervised G-SFDA (SSG-SFDA) for domain incremental learning, where a pre-trained source model (instead of the source data), few labeled target data, and plenty of unlabeled target data, are available. The goal is to achieve good performance on all domains. To cope with domain-ID agnostic, SSG-SFDA trains a conditional variational auto-encoder (CVAE) for each domain to learn its feature distribution, and a domain discriminator using virtual shallow features generated by CVAE to estimate the domain ID. To cope with catastrophic forgetting, SSG-SFDA uses soft domain attention to improve the sparse domain attention in G-SFDA. To cope with insufficient labeled target data, SSG-SFDA uses MixMatch to augment the unlabeled target data and better exploit the few labeled target data. Experiments on three datasets demonstrated the effectiveness of SSG-SFDA.

Emotion recognition is an important part of affective computing. Utilizing information from multiple modalities would facilitate more accurate emotion recognition. The performance of data-driven machine learning models usually relies on a large amount of labeled training data. However, labeling emotional data is expensive, because each sample usually requires multiple evaluators to annotate. To alleviate the annotation cost, this paper proposes a cross-modal diversity measure that considers the correlation between different modalities and integrates it with the representativeness for sample selection in unsupervised active learning (AL) for regression. To our knowledge, this challenging multi-modal unsupervised AL scenario has not been explored before: previous research only considered either unsupervised uni-modal AL or supervised multi-modal AL. Experiments on RECOLA and IEMOCAP datasets demonstrated the effectiveness of our proposed AL approach.

Bootstrap aggregating (Bagging) and boosting are two popular ensemble learning approaches, which combine multiple base learners to generate a composite model for more accurate and more reliable performance. They have been widely used in biology, engineering, healthcare, etc. This article proposes BoostForest, which is an ensemble learning approach using BoostTree as base learners and can be used for both classification and regression. BoostTree constructs a tree model by gradient boosting. It increases the randomness (diversity) by drawing the cut-points randomly at node splitting. BoostForest further increases the randomness by bootstrapping the training data in constructing different BoostTrees. BoostForest generally outperformed four classical ensemble learning approaches (Random Forest, Extra-Trees, XGBoost and LightGBM) on 35 classification and regression datasets. Remarkably, BoostForest tunes its parameters by simply sampling them randomly from a parameter pool, which can be easily specified, and its ensemble learning framework can also be used to combine many other base learners.

A brain–computer interface (BCI) enables a user to communicate directly with a computer using only the central nervous system. An affective BCI (aBCI) monitors and/or regulates the emotional state of the brain, which could facilitate human cognition, communication, decision-making, and health. The last decade has witnessed rapid progress in aBCI research and applications, but there does not exist a comprehensive and up-to-date tutorial on aBCIs. This tutorial fills the gap. It introduces first the basic concepts of BCIs and then, in detail, the individual components in a closed-loop aBCI system, including signal acquisition, signal processing, feature extraction, emotion recognition, and brain stimulation. Next, it describes three representative applications of aBCIs, i.e., cognitive workload recognition, fatigue estimation, and depression diagnosis and treatment. Several challenges and opportunities in aBCI research and applications, including brain signal acquisition, emotion labeling, diversity and size of aBCI datasets, algorithm comparison, negative transfer in emotion recognition, and privacy protection and security of aBCIs, are also explained.

With the development of sensors, more and more multimodal data are accumulated, especially in biomedical and bioinformatics fields. Therefore, multimodal data analysis becomes very important and urgent. In this study, we combine multi-kernel learning and transfer learning, and propose a feature-level multi-modality fusion model with insufficient training samples. To be specific, we firstly extend kernel Ridge regression to its multi-kernel version under the lp-norm constraint to explore complementary patterns contained in multimodal data. Then we use marginal probability distribution adaption to minimize the distribution differences between the source domain and the target domain to solve the problem of insufficient training samples. Based on epilepsy EEG data provided by the University of Bonn, we construct 12 multi-modality & transfer scenarios to evaluate our model. Experimental results show that compared with baselines, our model performs better on most scenarios.

A brain–computer interface (BCI) establishes a direct communication pathway between the human brain and a computer. It has been widely used in medical diagnosis, rehabilitation, education, entertainment, and so on. Most research so far focuses on making BCIs more accurate and reliable, but much less attention has been paid to their privacy. Developing a commercial BCI system usually requires close collaborations among multiple organizations, e.g., hospitals, universities, and/or companies. Input data in BCIs, e.g., electroencephalogram (EEG), contain rich privacy information, and the developed machine learning model is usually proprietary. Data and model transmission among different parties may incur significant privacy threats, and hence, privacy protection in BCIs must be considered. Unfortunately, there does not exist any contemporary and comprehensive review on privacy-preserving BCIs. This article fills this gap, by describing potential privacy threats and protection strategies in BCIs. It also points out several challenges and future research directions in developing privacy-preserving BCIs.

Recently, mini-batch gradient descent (MBGD)-based optimization has become popular in Takagi–Sugeno–Kang (TSK) fuzzy system optimization. However, it suffers from some challenges, including the curse of dimensionality and the sensitivity to the choice of the optimizer. The former has been alleviated by our previously proposed high-dimensional TSK (HTSK) algorithm. In this article, we point out that the latter is caused by the gradient vanishing problem on the rule consequent parameters, which in turn is caused by the small magnitude of the normalized rule firing levels, especially when the number of rules is large. Thus, the rule consequents are easily trapped into a bad local minimum with poor generalization performance. We propose to use first layer normalization (LN) to amplify the small firing levels, and then rectified linear unit (ReLU) to discard rules far away from the current training sample. We evaluated our proposed HTSK-LN and HTSK-LN-ReLU on twelve regression datasets with various sizes and dimensionalities. Experiments demonstrated that they can significantly improve the generalization performance, regardless of the training set size, feature dimensionality, choice of the optimizer, and rulebase size.

Acquisition of labeled training samples for affective computing is usually costly and time-consuming, as affects are intrinsically subjective, subtle and uncertain, and hence multiple human assessors are needed to evaluate each affective sample. Particularly, for affect estimation in the 3D space of valence, arousal and dominance, each assessor has to perform the evaluations in three dimensions, which makes the labeling problem even more challenging. Many sophisticated machine learning approaches have been proposed to reduce the data labeling requirement in various other domains, but so far few have considered affective computing. This paper proposes two multi-task active learning for regression approaches, which select the most beneficial samples to label, by considering the three affect primitives simultaneously. Experimental results on the VAM corpus demonstrated that our optimal sample selection approaches can result in better estimation performance than random selection and several traditional single-task active learning approaches. Thus, they can help alleviate the data labeling problem in affective computing, i.e., better estimation performance can be obtained from fewer labeling queries.

Emotion recognition is an important part of affective computing. Human emotions can be described categorically or dimensionally. Accurate machine learning models for emotion classification and estimation usually depend on a large amount of annotated data. However, label acquisition in emotion recognition is costly: obtaining the ground-truth labels of an emotional sample usually requires multiple annotators’ assessments, which is expensive and time-consuming. To reduce the labeling effort in multi-task emotions recognition, the paper proposes an inconsistency measure that can indicate the difference between the labels estimated from the feature space and the label distribution of labeled dataset. Using the inconsistency as an indicator of sample informativeness, we further propose an inconsistency-based multi-task cooperative learning framework that integrates multi-task active learning and self-training semi-supervised learning. Experiments in two multi-task emotion recognition scenarios, multi-dimensional emotion estimation and simultaneous emotion classification and estimation, were conducted under this framework. The results demonstrated that the proposed multi-task active learning framework outperformed several single-task and multi-task active learning approaches.

Network security requires effective detection and proper analysis of abnormal network behavior. To address the uncertainty associated with the process of network intrusion detection, this article proposes a network intrusion-detection algorithm based on dynamic intuitionistic fuzzy sets (IFSs). We use the classic network intrusion datasets KDD 99, NSL-KDD, and the massive, high-dimensional dataset UNSW-NB15 to evaluate the performance of our proposed algorithm. First, we perform data preprocessing on these three datasets and select features based on the results of a chi-square test. Second, using time-series processing, we construct dynamic intuitionistic fuzzy patterns from the feature-selected datasets. At last, we use a proposed distance measure for the dynamic IFSs to generate a classifier that facilitates the detection of network intrusion. Experimental results show that the classification performance of the proposed algorithm is superior to that of other state-of-the-art algorithms on the three aforementioned datasets. The achieved improvement in classification performance is particularly significant for large datasets.

Adaptive imagesteganography embeds secret messages into areas of cover images with complex features, including rich edges and complex textures. In this article, an adaptive image steganography technique based on the edge and complex texture areas of images is proposed, by comprehensively considering three rules in the design of image steganography. First, the embedding area is composed of the edge and complex texture areas of images, according to the complexity-first rule. Edge detection is realized by an improved fuzzy enhancement function, optimized by the grey wolf optimizer to detect both the weak and strong edges. Second, the minimum average classification error rate is used to assess the choice of the complex texture areas. Third, under the spreading rule, two different average filters and one KerBohme filter are used to design the cost function in the embedding areas. Finally, confidential information is adaptively embedded through syndrome-trellis codes. Experimental results show that the proposed algorithm outperforms seven classical adaptive image steganography algorithms on two steganalytic feature sets. The performance improvement is particularly significant when the payload is large.

Over-parameterized deep neural networks (DNNs) with sufficient capacity to memorize random noise can achieve excellent generalization performance, challenging the bias-variance trade-off in classical learning theory. Recent studies claimed that DNNs first learn simple patterns and then memorize noise; some other works showed a phenomenon that DNNs have a spectral bias to learn target functions from low to high frequencies during training. However, we show that the monotonicity of the learning bias does not always hold: under the experimental setup of deep double descent, the high-frequency components of DNNs diminish in the late stage of training, leading to the second descent of the test error. Besides, we find that the spectrum of DNNs can be applied to indicating the second descent of the test error, even though it is calculated from the training set only.

Takagi-Sugeno-Kang (TSK) fuzzy system with Gaussian membership functions (MFs) is one of the most widely used fuzzy systems in machine learning. However, it usually has difficulty handling high-dimensional datasets. This paper explores why TSK fuzzy systems with Gaussian MFs may fail on high-dimensional inputs. After transforming defuzzification to an equivalent form of softmax function, we find that the poor performance is due to the saturation of softmax. We show that two defuzzification operations, LogTSK and HTSK, the latter of which is first proposed in this paper, can avoid the saturation. Experimental results on datasets with various dimensionalities validated our analysis and demonstrated the effectiveness of LogTSK and HTSK.

Emotion recognition, which aims to identify an individual’s emotional state from the acquired physiological or body signals, is very important in affective computing. Emotions have two common representations: categorical, e.g., happy, sad, etc., and dimensional (continuous), e.g., valence, arousal and dominance. Training a good emotion classification or regression model usually requires a large number of labeled data. However, the labeling process is very difficult. As emotions are subtle and uncertain, it usually requires multiple assessors to label each emotional instance to obtain the groundtruth categorical label or dimensional values. In this paper, we propose a multi-task active learning (MTAL) framework to query the most useful samples for labeling, which enables the efficient training of an emotion classification model and multiple emotion regression models simultaneously. This is novel and challenging, as all previous research considered only emotion classification or regression alone, but not simultaneously. Experimental results on the IEMOCAP dataset demonstrated that MTAL outperformed random selection and several state-of-the-art single task active learning approaches, i.e., with the same number of labeled samples, MTAL can obtain better emotion classification and regression models simultaneously.

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment.Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications.EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas.In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years.Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals.Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications.Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.!

Traditional clustering algorithms for medical image segmentation can only achieve satisfactory clustering performance under relatively ideal conditions, in which there is adequate data from the same distribution, and the data is rarely disturbed by noise or outliers. However, a sufficient amount of medical images with representative manual labels are often not available, because medical images are frequently acquired with different scanners (or different scan protocols) or polluted by various noises. Transfer learning improves learning in the target domain by leveraging knowledge from related domains. Given some target data, the performance of transfer learning is determined by the degree of relevance between the source and target domains. To achieve positive transfer and avoid negative transfer, a negative-transfer-resistant mechanism is proposed by computing the weight of transferred knowledge. Extracting a negative-transfer-resistant fuzzy clustering model with a shared cross-domain transfer latent space (called NTR-FC-SCT) is proposed by integrating negative-transfer-resistant and maximum mean discrepancy (MMD) into the framework of fuzzy c-means clustering. Experimental results show that the proposed NTR-FC-SCT model outperformed several traditional non-transfer and related transfer clustering algorithms.

Vehicle accidents are the primary cause of fatalities worldwide. Most often, experiencing fatigue on the road leads to operator errors and behavioral lapses. Thus, there is a need to predict the cognitive state of drivers, particularly their fatigue level. Electroencephalography (EEG) has been demonstrated to be effective for monitoring changes in the human brain state and behavior. Thirty-seven subjects participated in this driving experiment and performed a perform lane-keeping task in a visual-reality environment. Three domains, namely, frequency, temporal, and 2-D spatial information, of the EEG channel location were comprehensively considered. A 4-D convolutional neural-network (4-D CNN) algorithm was then proposed to associate all information from the EEG signals and the changes in the human state and behavioral performance. A 4-D CNN achieves superior forecasting performance over 2-D CNN, 3-D CNN, and shallow networks. The results showed a 3.82% improvement in the root mean-square error, a 3.45% improvement in the error rate, and a 11.98% improvement in the correlation coefficient with 4-D CNN compared with 3-D CNN. The 4-D CNN algorithm extracts the significant theta and alpha activations in the frontal and posterior cingulate cortices under distinct fatigue levels. This work contributes to enhancing our understanding of deep learning methods in the analysis of EEG signals. We even envision that deep learning might serve as a bridge between translation neuroscience and further real-world applications.

Improving the efficiency of type-reduction algorithms continues to attract research interest. Recently, there has been some new type-reduction approaches claiming that they are more efficient than the well-known algorithms such as the enhanced Karnik–Mendel (EKM) and the enhanced iterative algorithm with stopping condition (EIASC). In a previous paper, we found that the computational efficiency of an algorithm is closely related to the platform, and how it is implemented. In computer science, the dependence on languages is usually avoided by focusing on the complexity of algorithms (using big O notation). In this article, the main contribution is the proposal of two novel type-reduction algorithms. Also, for the first time, a comprehensive study on both existing and new type-reduction approaches is made based on both algorithm complexity and practical computational time under a variety of programming languages. Based on the results, suggestions are given for the preferred algorithms in different scenarios depending on implementation platform and application context.

With the development of medical artificial intelligence, automatic magnetic resonance image (MRI) segmentation method is quite desirable. Inspired by the power of deep neural networks, a novel deep adversarial network, dilated block adversarial network (DBAN), is proposed to perform left ventricle, right ventricle, and myocardium segmentation in short-axis cardiac MRI. DBAN contains a segmentor along with a discriminator. In the segmentor, the dilated block (DB) is proposed to capture, and aggregate multi-scale features. The segmentor can produce segmentation probability maps while the discriminator can differentiate the segmentation probability map, and the ground truth at the pixel level. In addition, confidence probability maps generated by the discriminator can guide the segmentor to modify segmentation probability maps. Extensive experiments demonstrate that DBAN has achieved the state-of-the-art performance on the ACDC dataset. Quantitative analyses indicate that cardiac function indices from DBAN are similar to those from clinical experts. Therefore, DBAN can be a potential candidate for short-axis cardiac MRI segmentation in clinical applications.

In the field of intelligent transportation, transfer learning (TL) is often used to recognize EEG-based drowsy driving for a new subject with few subject-specific calibration data. However, most of existing TL-based models are offline, non-transparent, and in which features are only represented from one view (usually only one algorithm is used to extract features). In this paper, we consider an online multi-view regression model with high interpretability. By taking the 1-order TSK fuzzy system as the basic regression component and injecting the nature of the multi-view settings into the existing transfer learning framework and enforcing the consistencies across different views, we propose an online multi-view & transfer TSK fuzzy system for driver drowsiness estimation. In this novel model, features in both the source domain and the target domain are represented from multi-view perspectives such that more pattern information can be utilized during model training. Also, comparing with offline training, the proposed online fuzzy system meets the practical requirements more competently. An experiment on a driving dataset demonstrates that the proposed fuzzy system has smaller drowsiness estimation errors and higher interpretability than introduced benchmarking models.

In many real-world machine learning applications, unlabeled samples are easy to obtain, but it is expensive and/or time-consuming to label them. Active learning is a common approach for reducing this data labeling effort. It optimally selects the best few samples to label, so that a better machine learning model can be trained from the same number of labeled samples. This paper considers active learning for regression (ALR) problems. Three essential criteria - informativeness, representativeness, and diversity - have been proposed for ALR. However, very few approaches in the literature have considered all three of them simultaneously. We propose three new ALR approaches, with different strategies for integrating the three criteria. Extensive experiments on 12 datasets in various domains demonstrated their effectiveness.

Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approaches that preserve only the global data structure, such as principal component analysis (PCA), are usually sensitive to outliers. Approaches that preserve only the local data structure, such as locality preserving projections, are usually unsupervised (and hence cannot use label information) and uses a fixed similarity graph. We propose a novel linear dimensionality reduction approach, supervised discriminative sparse PCA with adaptive neighbors (SDSPCAAN), to integrate neighborhood-free supervised discriminative sparse PCA and projected clustering with adaptive neighbors. As a result, both global and local data structures, as well as the label information, are used for better dimensionality reduction. Classification experiments on nine high-dimensional datasets validated the effectiveness and robustness of our proposed SDSPCAAN.

Heart rate estimation from electrocardiogram signals is very important for the early detection of cardiovascular diseases. However, due to large individual differences and varying electrocardiogram signal quality, there does not exist a single reliable estimation algorithm that works well on all subjects. Every algorithm may break down on certain subjects, resulting in a significant estimation error. Ensemble regression, which aggregates the outputs of multiple base estimators for more reliable and stable estimates, can be used to remedy this problem. Moreover, active learning can be used to optimally select a few trials from a new subject to label, based on which a stacking ensemble regression model can be trained to aggregate the base estimators. This paper proposes four active stacking approaches, and demonstrates that they all significantly outperform three common unsupervised ensemble regression approaches, and a supervised stacking approach which randomly selects some trials to label. Remarkably, our active stacking approaches only need three or four labeled trials from each subject to achieve an average root mean squared estimation error below three beats per minute, making them very convenient for real-world applications. To our knowledge, this is the first research on active stacking, and its application to heart rate estimation.

Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the conditional distribution discrepancy; however, a more natural metric may be their joint probability distribution discrepancy. Additionally, most metrics only aim to increase the transferability between domains, but ignores the discriminability between different classes, which may result in insufficient classification performance. To address these issues, discriminative joint probability MMD (DJP-MMD) is proposed in this paper to replace the frequently-used joint MMD in domain adaptation. It has two desirable properties: 1) it provides a new theoretical basis for computing the distribution discrepancy, which is simpler and more accurate; 2) it increases the transferability and discriminability simultaneously. We validate its performance by embedding it into a joint probability domain adaptation framework. Experiments on six image classification datasets demonstrated that the proposed DJP-MMD can outperform traditional MMDs.

Takagi-Sugeno-Kang (TSK) fuzzy systems are flexible and interpretable machine learning models; however, they may not be easily optimized when the data size is large, and/or the data dimensionality is high. This article proposes a minibatch gradient descent (MBGD) based algorithm to efficiently and effectively train TSK fuzzy classifiers. It integrates two novel techniques: First, uniform regularization (UR), which forces the rules to have similar average contributions to the output, and hence to increase the generalization performance of the TSK classifier; and, second, batch normalization (BN), which extends BN from deep neural networks to TSK fuzzy classifiers to expedite the convergence and improve the generalization performance. Experiments on 12 UCI datasets from various application domains, with varying size and dimensionality, demonstrated that UR and BN are effective individually, and integrating them can further improve the classification performance.

Interacting with human operators, remote environment, and communication networks, teleoperation systems are considerably suffering from complexities and uncertainties. Managing these is of paramount importance for safe and smooth performance of teleoperation systems. Among the countless solutions developed by researchers, type-2 fuzzy (T2F) algorithms have shown an outstanding performance in modeling complex systems and tackling uncertainties. Moreover, artificial neural networks (NNs) are well known for their adaptive learning potentials. This article proposes an adaptive interval type-2 fuzzy neural-network control scheme for teleoperation systems with time-varying delays and uncertainties. The T2F models are developed based on the experimental data collected from a teleoperation setup over a local computer network. However, the resulted controller is evaluated on an intercontinental communication network through the Internet between Australia and Scotland. Moreover, the slave robot and the remote workspace are completely different and unforeseen. Stability and performance of the proposed control is analyzed by Lyapunov-Krasovskii method. Comprehensive comparative studies demonstrate that the proposed controller outperforms traditional techniques in experimental evaluations.

Fuzzy systems have achieved great success in numerous applications. However, there are still many challenges in designing an optimal fuzzy system, e.g., how to efficiently optimize its parameters, how to balance the trade-off between cooperations and competitions among the rules, how to overcome the curse of dimensionality, how to increase its generalization ability, etc. Literature has shown that by making appropriate connections between fuzzy systems and other machine learning approaches, good practices from other domains may be used to improve the fuzzy systems, and vice versa. This article gives an overview on the functional equivalence between Takagi-Sugeno-Kang fuzzy systems and four classic machine learning approaches-neural networks, mixture of experts, classification and regression trees, and stacking ensemble regression-for regression problems. We also point out some promising new research directions, inspired by the functional equivalence, that could lead to solutions to the aforementioned problems. To our knowledge, this is so far the most comprehensive overview on the connections between fuzzy systems and other popular machine learning approaches, and hopefully will stimulate more hybridization between different machine learning algorithms.

There have been different strategies to improve the performance of a machine learning model, e.g., increasing the depth, width, and/or nonlinearity of the model, and using ensemble learning to aggregate multiple base/weak learners in parallel or in series. This article proposes a novel strategy called patch learning (PL) for this problem. It consists of three steps: first, train an initial global model using all training data; second, identify from the initial global model the patches that contribute the most to the learning error, and train a (local) patch model for each such patch; and, third, update the global model using training data that do not fall into any patch. To use a PL model, we first determine if the input falls into any patch. If yes, then the corresponding patch model is used to compute the output. Otherwise, the global model is used. We explain in detail how PL can be implemented using fuzzy systems. Five regression problems on one-dimensional (1-D)/2-D/3-D curve fitting, nonlinear system identification, and chaotic time-series prediction, verified its effectiveness. To our knowledge, the PL idea has not appeared in the literature before, and it opens up a promising new line of research in machine learning.

Multitask learning uses auxiliary data or knowledge from relevant tasks to facilitate the learning in a new task. Multitask optimization applies multitask learning an optimization to study how effectively and efficiently tackle the multiple optimization problems, simultaneously. Evolutionary multitasking, or multifactorial optimization, is an emerging subfield of multitask optimization, which integrates evolutionary computation and multitask learning. This article proposes a novel and easy-to-implement multitasking genetic algorithm (MTGA), which copes well with significantly different optimization tasks by estimating and using the bias among them. Comparative studies with eight state-of-the-art single-task and multitask approaches in the literature on nine benchmarks demonstrated that, on average, the MTGA outperformed all of them and had lower computational cost than six of them. Based on the MTGA, a simultaneous optimization strategy for fuzzy system design is also proposed. Experiments on simultaneous optimization of type-1 and interval type-2 fuzzy logic controllers for couple-tank water level control demonstrated that the MTGA can find better fuzzy logic controllers than other approaches.

Takagi–Sugeno–Kang (TSK) fuzzy systems are very useful machine learning models for regression problems. However, to our knowledge, there has not existed an efficient and effective training algorithm that ensures their generalization performance and also enables them to deal with big data. Inspired by the connections between TSK fuzzy systems and neural networks, we extend three powerful neural network optimization techniques, i.e., minibatch gradient descent (MBGD), regularization, and AdaBound, to TSK fuzzy systems, and also propose three novel techniques (DropRule, DropMF, and DropMembership) specifically for training TSK fuzzy systems. Our final algorithm, MBGD with regularization, DropRule, and AdaBound, can achieve fast convergence in training TSK fuzzy systems, and also superior generalization performance in testing. It can be used for training TSK fuzzy systems on datasets of any size; however, it is particularly useful for big datasets, on which currently no other efficient training algorithms exist.

Objective: This paper aims at reducing the calibration effort of EEG-based brain-computer interfaces (BCIs). More specifically, in the context of cross-subject classification, we correct covariate shift of EEG data from different subjects, so that a classifier trained on auxiliary subjects can also be applied to a new subject, without any labeled trials from the new subject. Methods: We propose two approaches to enhance the performance of a state-of-the-art Riemannian space transfer learning (TL) algorithm: 1) trials selection, which resamples trials from the auxiliary subjects so that they become more consistent with those of the new subject; and, 2) channel selection, which reduces the number of channels and hence makes the Riemannian space computations more accurate and efficient. Results: We tested the proposed approaches on two motor imagery datasets. The results verified that they can enhance the performance of the state-of-the-art TL algorithm. Conclusion and significance: Our proposed approaches make the state-of-the-art TL algorithm more effective and efficient.

The similarity between two fuzzy sets (FSs) is an important concept in fuzzy logic. As the research interest on general type-2 (GT2) FSs has increased recently, many similarity measures for them have also been proposed. This paper gives a comprehensive overview of existing similarity measures for GT2 FSs, points out their limitations, and, by using an intuitive geometric explanation, proposes a Jaccard similarity measure for GT2 FSs that is an extension of the popular Jaccard similarity measure for type-1 and interval type-2 FSs. The fundamental difference between the proposed Jaccard similarity measure for GT2 FSs and all existing similarity measures is that the Jaccard similarity measure considers the overall geometries of two GT2 FSs and does not depend on a specific representation of the GT2 FSs, whereas all existing similarity measures for GT2 FSs depend either on the vertical slice representation or the α-plane representation. We show that the Jaccard similarity measure for GT2 FSs satisfies four properties of a similarity measure and demonstrate its reasonableness using two examples.

The representation theorem for interval type-2 fuzzy sets (IT2 FSs), proposed by Mendel and John, states that an IT2 FS is a combination of all its embedded type-1 (T1) FSs, which can be nonconvex and/or subnormal. These nonconvex and/or subnormal embedded T1 FSs are included in developing many theoretical results for IT2 FSs, including uncertainty measures, the linguistic weighted averages (LWAs), the ordered LWAs (OLWAs), the linguistic weighted power means (LWPMs), etc. However, convex and normal T1 FSs are used in most fuzzy logic applications, particularly computing with words. In this paper, we propose a constrained representation theorem (CRT) for well-shaped IT2 FSs using only its convex and normal embedded T1 FSs, and show that IT2 FSs generated from three word encoding approaches and four computing with words engines (LWAs, OLWAs, LWPMs, and perceptual reasoning) are all well-shaped IT2 FSs. We also compute five constrained uncertainty measures (centroid, cardinality, fuzziness, variance, and skewness) for well-shaped IT2 FSs using the CRT. The CRT and the associated constrained uncertainty measures can be useful in computing with words, IT2 fuzzy logic system design using the principles of uncertainty, and measuring the similarity between two well-shaped IT2 FSs.

Multiview datasets are frequently encountered in learning tasks, such as web data mining and multimedia information analysis. Given a multiview dataset, traditional learning algorithms usually decompose it into several single-view datasets, from each of which a single-view model is learned. In contrast, a multiview learning algorithm can achieve better performance by cooperative learning on the multiview data. However, existing multiview approaches mainly focus on the views that are visible and ignore the hidden information behind the visible views, which usually contains some intrinsic information of the multiview data, or vice versa. To address this problem, this paper proposes a multiview fuzzy logic system which utilizes both the hidden information shared by the multiple visible views and the information of each visible view. Extensive experiments were conducted to validate its effectiveness.

Active learning (AL) is a machine-learning approach for reducing the data labeling effort. Given a pool of unlabeled samples, it tries to select the most useful ones to label so that a model built from them can achieve the best possible performance. This paper focuses on pool-based sequential AL for regression (ALR). We first propose three essential criteria that an ALR approach should consider in selecting the most useful unlabeled samples: informativeness, representativeness, and diversity, and compare four existing ALR approaches against them. We then propose a new ALR approach using passive sampling, which considers both the representativeness and the diversity in both the initialization and subsequent iterations. Remarkably, this approach can also be integrated with other existing ALR approaches in the literature to further improve the performance. Extensive experiments on 11 University of California, Irvine, Carnegie Mellon University StatLib, and University of Florida Media Core data sets from various domains verified the effectiveness of our proposed ALR approaches.

Many machine learning (ML) applications rely on large amounts of personal data for training and inference. Among the most intimate exploited data sources is electroencephalogram (EEG) data. The emergence of consumer -grade, low-cost brain -computer interfaces (BCIs) and corresponding software development kits' is bringing the use of BCI within reach of application developers. The access that BCI applications have to neural signals rightly raises privacy concerns. Application developers can easily gain knowledge beyond the professed scope from unprotected EEG signals, including passwords, ATM PINs, and other personal data. The challenge is how to engage in meaningful ML with EEG data while protecting the privacy of users.

This paper proposes applying recurrent residual network (RRN) for analyzing electroencephalogram (EEG) data captured during a simulated sustained attention driving task. We first address the suitableness of utilizing residual structure as well as adopting recurrent structure for EEG signal processing. Then based on these descriptions a recurrent residual network is tailored and depicted in detail. Thirdly we use an EEG dataset obtained from a sustained-attention experiment for our model justification. By applying the RRN model to the experimental data and via the competitive result achieved, we demonstrate the elegance of the proposed model. At last, we discuss the characteristics of the learned filters and their interpretations from EEG frequency band perspectives.

This letter is a supplement to the previous paper “A Direct Approach for Determining the Switch Points in the Karnik-Mendel Algorithm”. In the previous paper, the enhanced iterative algorithm with stop condition (EIASC) was shown to be the most inefficient in R. Such outcome is apparently different from the results in another paper in which EIASC was illustrated to be the most efficient in MATLAB. An investigation has been made into this apparent inconsistency and it can be confirmed that both the results in R and MATLAB are valid for the EIASC algorithm. The main reason for such phenomenon is the efficiency difference of loop operations in R and MATLAB. It should be noted that the efficiency of an algorithm is closely related to its implementation in practice. In this letter, we update the comparisons of the three algorithms in the previous paper, based on optimized implementations under five programming languages (MATLAB, R, Python, C, and Java). From this, we conclude that results in one programming language cannot be simply extended to all languages.

This paper presents an integrated interval type-2 fuzzy logic approach that simultaneously models and controls an underactuated mobile two-wheeled inverted pendulum (MTWIP), which suffers from modeling uncertainties and external disturbances. The control objective is to attain the desired position and direction while keeping the MTWIP balanced. It is achieved by integrating four interval type-2 fuzzy logic systems (IT2 FLSs): the first IT2 FLS describes the dynamics of the MTWIP using a Takagi-Sugeno model, the second IT2 FLS controls the balance of the MTWIP using also a Takagi-Sugeno model, and the third and fourth IT2 FLSs control its position and direction, respectively, using a Mamdani model. A linear matrix inequality based design approach is also proposed to guarantee the stability of the balance controller. The proposed approach is compared with a type-1 FLS in real-world experiments. All results demonstrate that the IT2 FLS outperforms the type-1 FLS, especially under modeling uncertainties and external disturbances.

Electroencephalogram (EEG) signals are frequently used in brain-computer interfaces (BC!s), but they are easily contaminated by artifacts and noise, so preprocessing must be done before they are fed into a machine learning algorithm for classification or regression. Spatial filters have been widely used to increase the signal-to-noise ratio of EEG for BC! classification problems, but their applications in BC! regression problems have been very limited. This paper proposes two common spatial pattern (CSP) filters for EEG-based regression problems in BC!, which are extended from the CSP filter for classification, by using fuzzy sets. Experimental results on EEG-based response speed estimation from a large-scale study, which collected 143 sessions of sustained-attention psychomotor vigilance task data from 17 subjects during a 5-month period, demonstrate that the two proposed spatial filters can significantly increase the EEG signal quality. When used in LASSO and k-nearest neighbors regression for user response speed estimation, the spatial filters can reduce the root-mean-square estimation error by 10.02-19.77%, and at the same time increase the correlation to the true response speed by 19.39-86.47%.

Brain-state drifts could significantly impact on the performance of machine-learning algorithms in brain computer interface (BCI). However, less is understood with regard to how brain transition states influence a model and how it can be represented for a system. Herein we are interested in the hidden information of brain state-drift occurring in both simulated and real-world human-system interaction. This research introduced the Riemann metric to categorize EEG data, and visualized the clustering result so that the distribution of the data can be observable. Moreover, to defeat subjective uncertainty of electroencephalography (EEG) signals, fuzzy theory was employed. In this study, we built a fuzzy rule-based brain-state-drift detector to observe the brain state and imported data from different subjects to testify the performance. The result of the detection is acceptable and shown in this paper. In the future, we expect that brain-state drifting can be connected with human behaviors via the proposed fuzzy rule-based classification. We also will develop a new structure for a fuzzy rule-based brain-state-drift detector to improve the detection accuracy.

Single-trial classification of event-related potentials in electroencephalogram (EEG) signals is a very important paradigm of brain-computer interface (BCI). Because of individual differences, usually some subject-specific calibration data are required to tailor the classifier for each subject. Transfer learning has been extensively used to reduce such calibration data requirement, by making use of auxiliary data from similar/relevant subjects/tasks. However, all previous research assumes that all auxiliary data have been labeled. This paper considers a more general scenario, in which part of the auxiliary data could be unlabeled. We propose active semi-supervised transfer learning (ASTL) for offline BCI calibration, which integrates active learning, semi-supervised learning, and transfer learning. Using a visual evoked potential oddball task and three different EEG headsets, we demonstrate that ASTL can achieve consistently good performance across subjects and headsets, and it outperforms some state-of-the-art approaches in the literature.

This letter provides a critical review of “A New Look at Type-2 Fuzzy Sets and Type-2 Fuzzy Logic Systems” IEEE Trans. Fuzzy Systems, and debunks its four claims.

One big challenge that hinders the transition of brain-computer interfaces (BCIs) from laboratory settings to real-life applications is the availability of high-performance and robust learning algorithms that can effectively handle individual differences, i.e., algorithms that can be applied to a new subject with zero or very little subject-specific calibration data. Transfer learning and domain adaptation have been extensively used for this purpose. However, most previous works focused on classification problems. This paper considers an important regression problem in BCI, namely, online driver drowsiness estimation from EEG signals. By integrating fuzzy sets with domain adaptation, we propose a novel online weighted adaptation regularization for regression (OwARR) algorithm to reduce the amount of subject-specific calibration data, and also a source domain selection (SDS) approach to save about half of the computational cost of OwARR. Using a simulated driving dataset with 15 subjects, we show that OwARR and OwARR-SDS can achieve significantly smaller estimation errors than several other approaches. We also provide comprehensive analyses on the robustness of OwARR and OwARR-SDS.

The papers in this special section focus on brain computer interfaces. A brain computer interface (BCI) enables direct communication between the brain and a computer. It can be used to research, repair, or enhance human cognitive or sensorymotor functions. BCIs have attracted rapidly increasing research interest in the last decade, thanks to recent advances in neurosciences, wearable/mobile biosensors, and analytics. However, there are many challenges in the transition from laboratory settings to real-life applications, including the reliability and convenience of the sensing hardware, the availability of high performance and robust algorithms for signal analysis and interpretation, and fundamental advances in automated reasoning that enable the reasoning and generalization across individuals. Computational intelligence techniques, particularly fuzzy sets and systems, have demonstrated outstanding performance in handling uncertainties in many real-world applications.

Many real-world brain–computer interface (BCI) applications rely on single-trial classification of event-related potentials (ERPs) in EEG signals. However, because different subjects have different neural responses to even the same stimulus, it is very difficult to build a generic ERP classifier whose parameters fit all subjects. The classifier needs to be calibrated for each individual subject, using some labeled subject-specific data. This paper proposes both online and offline weighted adaptation regularization (wAR) algorithms to reduce this calibration effort, i.e., to minimize the amount of labeled subject-specific EEG data required in BCI calibration, and hence to increase the utility of the BCI system. We demonstrate using a visually evoked potential oddball task and three different EEG headsets that both online and offline wAR algorithms significantly outperform several other algorithms. Moreover, through source domain selection, we can reduce their computational cost by about$\text{50}\%$, making them more suitable for real-time applications.

Ensemble learning is a powerful approach to construct a strong learner from multiple base learners. The most popular way to aggregate an ensemble of classifiers is majority voting, which assigns a sample to the class that most base classifiers vote for. However, improved performance can be obtained by assigning weights to the base classifiers according to their accuracy. This paper proposes an agreement rate initialized maximum likelihood estimator (ARIMLE) to optimally fuse the base classifiers. ARIMLE first uses a simplified agreement rate method to estimate the classification accuracy of each base classifier from the unlabeled samples, then employs the accuracies to initialize a maximum likelihood estimator (MLE), and finally uses the expectation-maximization algorithm to refine the MLE. Extensive experiments on visually evoked potential classification in a brain-computer interface application show that ARIMLE outperforms majority voting, and also achieves better or comparable performance with several other state-of-the-art classifier combination approaches.

There are many important regression problems in real-world brain-computer interface (BCI) applications, e.g., driver drowsiness estimation from EEG signals. This paper considers offline analysis: given a pool of unlabeled EEG epochs recorded during driving, how do we optimally select a small number of them to label so that an accurate regression model can be built from them to label the rest? Active learning is a promising solution to this problem, but interestingly, to our best knowledge, it has not been used for regression problems in BCI so far. This paper proposes a novel enhanced batch-mode active learning (EBMAL) approach for regression, which improves upon a baseline active learning algorithm by increasing the reliability, representativeness and diversity of the selected samples to achieve better regression performance. We validate its effectiveness using driver drowsiness estimation from EEG signals. However, EBMAL is a general approach that can also be applied to many other offline regression problems beyond BCI.

To facilitate the transition of brain-computer interface (BCI) systems from laboratory settings to real-world application, it is very important to minimize or even completely eliminate the subject-specific calibration requirement. There has been active research on calibrationless BCI systems for classification applications, e.g., P300 speller. To our knowledge, there is no literature on calibrationless BCI systems for regression applications, e.g., estimating the continuous drowsiness level of a driver from EEG signals. This paper proposes a novel spectral meta-learner for regression (SMLR) approach, which optimally combines base regression models built from labeled data from auxiliary subjects to label offline EEG data from a new subject. Experiments on driver drowsiness estimation from EEG signals demonstrate that SMLR significantly outperforms three state-of-the-art regression model fusion approaches. Although we introduce SMLR as a regression model fusion in the BCI domain, we believe its applicability is far beyond that.

Drowsy driving is a pervasive problem among drivers, and is also an important contributor to motor vehicle accidents. It is very important to be able to estimate a driver's drowsiness level online so that preventative actions could be taken to avoid accidents. However, because of large individual differences, it is very challenging to design an estimation algorithm whose parameters fit all subjects. Some subject-specific calibration data must be used to tailor the algorithm for each new subject. This paper proposes a domain adaptation with model fusion (DAMF) online drowsiness estimation approach using EEG signals. By making use of EEG data from other subjects in a transfer learning framework, DAMF requires very little subject-specific calibration data, which significantly increases its utility in practice. We demonstrate using a simulated driving experiment and 15 subjects that DAMF can achieve much better performance than several other approaches.

Rapid serial visual presentation based brain-computer interface (BCI) system relies on single-trial classification of event-related potentials. Because of large individual differences, some labeled subject-specific data are needed to calibrate the classifier for each new subject. This paper proposes an online weighted adaptation regularization (OwAR) algorithm to reduce the online calibration effort, and hence to increase the utility of the BCI system. We show that given the same number of labeled subject-specific training samples, OwAR can significantly improve the online calibration performance. In other words, given a desired classification accuracy, OwAR can significantly reduce the number of labeled subject-specific training samples. Furthermore, we also show that the computational cost of OwAR can be reduced by more than 50% by source domain selection, without a statistically significant sacrifice of classification performance.

As an integral part of interval type-2 fuzzy logic system (IT2FLS), type reduction (TR) plays a vital role in determining the performance of IT2FLS. Out of many type reduction algorithms, only Karnik-Mendel type TR algorithms capture the essence of interval type-2 fuzzy sets in type reduction. Enhanced Karnik-Mendel (EKM) algorithm is the most commonly used TR algorithm. In this work, we propose three new initializations for EKM algorithm. It is shown they are performing better than EKM and one of the proposed initializations significantly outperforms others. The performance gain can be upto 40% as per comprehensive simulation results demonstrated in this paper. Our findings are justified by computational time savings and iteration requirement for switch point search.

Karnik-Mendel (KM) algorithm is the most used and researched type reduction (TR) algorithm in literature. This algorithm is iterative in nature and despite consistent long term effort, no general closed form formula has been found to replace this computationally expensive algorithm. In this research work, we demonstrate that the outcome of KM algorithm can be approximated by simple linear regression techniques. Since most of the applications will have a fixed range of inputs with small scale variations, it is possible to handle those complexities in design phase and build a fuzzy logic system (FLS) with low run time computational burden. This objective can be well served by the application of regression techniques. This work presents an overview of feasibility of regression techniques for design of data-driven type reducers while keeping the uncertainty bound in FLS intact Simulation results demonstrates the approximation error is less than 2%. Thus our work preserve the essence of Karnik-Mendel algorithm and serves the requirement of low computational complexities.

Karnik-Mendel (KM) algorithm is the most widely used type reduction (TR) method in literature for the design of interval type-2 fuzzy logic systems (IT2FLS). Its iterative nature for finding left and right switch points is its Achilles heel. Despite a decade of research, none of the alternative TR methods offer uncertainty measures equivalent to KM algorithm. This paper takes a data-driven approach to tackle the computational burden of this algorithm while keeping its key features. We propose a regression method to approximate left and right switch points found by KM algorithm. Approximator only uses the firing intervals, rules centroids, and FLS structural features as inputs. Once training is done, it can precisely approximate the left and right switch points through basic vector multiplications. Comprehensive simulation results demonstrate that the approximation accuracy for a wide variety of FLSs is 100%. Flexibility, ease of implementation, and speed are other features of the proposed method.

Electroencephalography (EEG) has been widely used in a variety of contexts, including medical monitoring of subjects as well as performance monitoring in healthy individuals. Recent technological advances have now enabled researchers to quickly record and collect EEG on a wide scale. Although EEG is fairly easy to record, it is highly susceptible to noise sources called artifacts which can occur at amplitudes several times greater than the EEG signal of interest. Because of this, users must manually annotate the EEG signal to identify artifact regions in the data prior to any downstream processing. This can be time-consuming and impractical for large data collections. In this paper we present a method which uses Active Learning (AL) to improve the reliability of existing EEG artifact classifiers with minimal amounts of user interaction. Our results show that classification accuracy equivalent to classifiers trained on full data annotation can be obtained while labeling less than 25% of the data. This suggests significant time savings can be obtained when manually annotating artifacts in large EEG data collections.

Single-trial classification of Event-Related Potentials (ERPs) is needed in many real-world brain-computer interface (BCI) applications. However, because of individual differences, the classifier needs to be calibrated by using some labeled subject specific training samples, which may be inconvenient to obtain. In this paper we propose a weighted adaptation regularization (wAR) approach for offline BCI calibration, which uses data from other subjects to reduce the amount of labeled data required in offline single-trial classification of ERPs. Our proposed model explicitly handles class-imbalance problems which are common in many real-world BCI applications. War can improve the classification performance, given the same number of labeled subject-specific training samples, or, equivalently, it can reduce the number of labeled subject-specific training samples, given a desired classification accuracy. To reduce the computational cost of wAR, we also propose a source domain selection (SDS) approach. Our experiments show that wARSDS can achieve comparable performance with wAR but is much less computationally intensive. We expect wARSDS to find broad applications in offline BCI calibration.

This paper provides a new methodology for determining a word's interval type-2 fuzzy set model using only one subject, a Person FOU. It uses interval end-point uncertainty intervals instead of only the end-point intervals. Such uncertainty intervals are relatively easy to collect and they do not introduce methodological uncertainties during the data-collection process. This new method is applied to ten probability words. Person FOUs are obtained for these words, and the robustness of this new method to the choice of the probability distribution that is assigned to the interval end-point uncertainty intervals is examined and demonstrated.

Interval type-2 fuzzy logic systems (IT2 FLSs) have become increasingly popular in the last decade, and have demonstrated superior performance in a number of applications. However, the computations in an IT2 FLS are more complex than those in a type-1 FLS, and there are many choices to be made in designing an IT2 FLS, including the shape of membership functions (Gaussian or trapezoidal), number of membership functions, type of fuzzifier (singleton or non-singleton), kind of rules (Mamdani or Takagi-Sugeno-Kang), type of i-norm (minimum or product), method to compute the output (type-reduction or not), and methods for tuning the parameters (gradient-based methods or evolutionary computation algorithms; one-step or two-step). While these choices give an experienced IT2 FLS researcher extensive freedom to design the optimal IT2 FLS, they may look overwhelming and confusing to IT2 beginners. Such a beginner may make an inappropriate choice, obtain unexpected results, and lose interest, which will hinder the wider applications of IT2 FLSs. In this paper we try to help IT2 beginners navigate through the maze by recommending some representative choices for an IT2 FLS design. We also clarify two myths about IT2 FLSs. This paper will make IT2 FLSs more accessible to IT2 beginners.

Single-trial Event-Related Potential (ERP) classification is a key requirement for several types of Brain-Computer Interaction (BCI) technologies. However, strong individual differences make it challenging to develop a generic single-trial ERP classifier that performs well for all subjects. Usually some subject-specific training samples need to be collected in an initial calibration session to customize the classifier. However, if implemented into an actual BCI system, then this calibration process would decrease the utility of the system, potentially decreasing its usability. In this paper we propose a Transfer Learning approach for reducing the amount of subject-specific data in online single-trial ERP classifier calibration, and an Active Transfer Learning approach for offline calibration. By applying these approaches to data from a Visually-Evoked Potential EEG experiment, we demonstrate that they improve the classification performance, given the same number of labeled subject-specific training samples. In other words, these approaches can also attain a desired level of classification accuracy with less labeling effort when compared to a randomly selected training set.

Interval type-2 fuzzy logic systems (IT2 FLSs) have demonstrated better abilities to handle uncertainties than their type-1 (T1) counterparts in many applications; however, the high computational cost of the iterative Karnik-Mendel (KM) algorithms in type-reduction means that it is more expensive to deploy IT2 FLSs, which may hinder them from certain cost-sensitive real-world applications. This paper provides a comprehensive overview and comparison of three categories of methods to reduce their computational cost. The first category consists of five enhancements to the KM algorithms, which are the most popular type-reduction algorithms to date. The second category consists of 11 alternative type-reducers, which have closed-form representations and, hence, are more convenient for analysis. The third category consists of a simplified structure for IT2 FLSs, which can be combined with any algorithms in the first or second category for further computational cost reduction. Experiments demonstrate that almost all methods in these three categories are faster than the KM algorithms. This overview and comparison will help researchers and practitioners on IT2 FLSs choose the most suitable structure and type-reduction algorithms, from a computational cost perspective. A recommendation is given in the conclusion.

The Word decoder is a very important approach for decoding in the Perceptual Computer. It maps the computing with words (CWW) engine output, which is a fuzzy set, into a word in a codebook so that it can be understood. However, the Word decoder suffers from significant information loss, i.e., the fuzzy set model of the mapped word may be quite different from the fuzzy set output by the CWW engine, especially when the codebook is small. In this paper we propose a Reconstruction decoder, which represents the CWW engine output as a combination of two successive codebook words with minimum information loss by solving a constrained optimization problem. The Reconstruction decoder can be viewed as a generalized Word decoder and it is also implicitly a Rank decoder. Moreover, it preserves the shape information of the CWW engine output in a simple form without sacrificing much accuracy. Experimental results verify the effectiveness of the Reconstruction decoder. Its Matlab implementation is also given in this paper.

Affective computing is computing that relates to, arises from, or deliberately influences emotions. It has lots of applications in the next generation of human-computer interfaces. We have proposed a closed-loop affective computing system, which includes affect recognition, affect modeling, and affect control. Because emotions include both intra-personal uncertainty, which is the uncertainty a person has about an emotion, and inter-personal uncertainty, which results from the fact that different people have different perceptions and expressions of the same emotion, it is promising to use fuzzy sets and systems, especially type-2 fuzzy sets and systems, to handle these uncertainties in an affective computing system. This paper introduces four applications of affective computing, reviews some recent advances on the application of fuzzy sets and systems to affect recognition, modeling, and control, and points out some new research directions. It will be very useful to both the fuzzy logic research community and the affective computing research community, especially to researchers working at the intersection of these two areas.

Interval type-2 fuzzy logic controllers (IT2 FLCs) have been attracting great research interests recently. There are many decisions to be made in designing an IT2 FLC. One of them is to determine which membership function type to use, e.g., Gaussian or trapezoidal. There have not been comprehensive studies on this problem so far. In this paper we present 12 considerations in choosing between Gaussian and trapezoidal membership functions for an IT2 FLC, including representation, construction, optimization, adaptiveness, novelty, analytical structure, continuity, monotonicity, stability, robustness, computational cost, and control performance. It can help practitioners select the appropriate membership function type in IT2 FLC design, and researchers identify new research opportunities on IT2 FLCs. Our study shows that each MF type has its own advantages: Gaussian IT2 FLCs are simpler in design because they are easier to represent and optimize, always continuous, and faster for small rulebases, whereas trapezoidal IT2 FLCs are simpler in analysis.

Interval type-2 fuzzy logic controllers have demonstrated better abilities to handle uncertainties than their type-1 counterparts in many applications; however, the high computational cost of the iterative Karnik-Mendel algorithms in type-reduction may hinder them from certain real-time applications. This paper provides an overview and comparison of 11 alternative type-reducers, which have closed-form representations and are more convenient in analysis. Experiments demonstrate that 10 of them are faster than the Karnik-Mendel algorithms. Among them, the Wu-Tan and Nie-Tan methods are the fastest, and they are only about 1.2-1.7 times slower than a type-1 fuzzy logic controller.

Interval type-2 fuzzy logic controllers (IT2 FLCs) have recently been attracting a lot of research attention. Many reported results have shown that IT2 FLCs are better able to handle uncertainties than their type-1 (T1) counterparts. A challenging question is the following: What are the fundamental differences between IT2 and T1 FLCs? Once the fundamental differences are clear, we can better understand the advantages of IT2 FLCs and, hence, make better use of them. This paper explains two fundamental differences between IT2 and T1 FLCs: 1) Adaptiveness, meaning that the embedded T1 fuzzy sets used to compute the bounds of the type-reduced interval change as input changes; and 2) Novelty, meaning that the upper and lower membership functions of the same IT2 fuzzy set may be used simultaneously in computing each bound of the type-reduced interval. T1 FLCs do not have these properties; thus, a T1 FLC cannot implement the complex control surface of an IT2 FLC given the same rulebase. We also present several methods to visualize and analyze the effects of these two fundamental differences, including the control surface, the P-map, the equivalent generalized T1 fuzzy sets, and the equivalent PI gains. Finally, we examine five alternative type reducers for IT2 FLCs and explain why they do not capture the fundamentals of IT2 FLCs.

Construction of interval type-2 fuzzy set models is the first step in the perceptual computer, which is an implementation of computing with words. The interval approach (IA) has, so far, been the only systematic method to construct such models from data intervals that are collected from a survey. However, as pointed out in this paper, it has some limitations, and its performance can be further improved. This paper proposes an enhanced interval approach (EIA) and demonstrates its performance on data that are collected from a web survey. The data part of the EIA has more strict and reasonable tests than the IA, and the fuzzy set part of the EIA has an improved procedure to compute the lower membership function. We also perform a convergence analysis to answer two important questions: 1) Does the output interval type-2 fuzzy set from the EIA converge to a stable model as increasingly more data intervals are collected, and 2) if it converges, then how many data intervals are needed before the resulting interval type-2 fuzzy set is sufficiently similar to the model obtained from infinitely many data intervals? We show that the EIA converges in a mean-square sense, and generally, 30 data intervals seem to be a good compromise between cost and accuracy.

In this paper, we present a solution to Zadeh's Magnus challenge problem on linguistic probabilities. First, we implement Zadeh's solution to this problem. Then, we use the intersection-product syllogism and a syllogism based on the entailment principle to interpret the problem so that it can be solved via Linguistic Weighted Averages. We show that the problem can be solved by calculation of pessimistic (lower) and optimistic (upper) probabilities via Linguistic Weighted Averages. Then, we choose vocabularies for quantifiers and linguistic probabilities that are involved in the problem statement. The vocabularies are modeled using interval type-2 fuzzy sets. We calculate optimistic (upper) and pessimistic (lower) probabilities, which naturally would be interval type-2 fuzzy sets. Finally, we map the pessimistic and optimistic probabilities to linguistic probabilities present in the vocabularies, so that the results can be comprehended by a human.

Type-reduction algorithms are very important for type-2 fuzzy sets and systems. The earliest one, and also the most popular one, is the Karnik-Mendel Algorithm, which is iterative and computationally intensive. In the last a few years researchers have proposed several other more efficient type-reduction algorithms. In this paper we also propose a new algorithm which improves over the latest results. Experiments show that it is the most efficient one to use in practice. Particularly, when the number of elements in type-reduction is smaller than 100, which is true in most practical type-reduction computations, our proposed algorithm can save over 50% computational cost over the Karnik Mendel Algorithms. We also give the Matlab implementation of our most efficient algorithm in the Appendix. It includes preprocessing steps to eliminate numerical problems, and also improved testing criteria to prevent possible infinite loops. This program will be very helpful in promoting the popularity of type 2 fuzzy sets and systems.

Linguistic summarization (LS) is a data mining or knowledge discovery approach to extract patterns from databases. Many authors have used this technique to generate summaries like “Most senior workers have high salary,” which can be used to better understand and communicate about data; however, few of them have used it to generate IF-THEN rules like “IFXis large andYis medium, THENZis small,” which not only facilitate understanding and communication of data but can also be used in decision-making. In this paper, an LS approach to generate IF-THEN rules for causal databases is proposed. Both type-1 and interval type-2 fuzzy sets are considered. Five quality measures-the degrees of truth, sufficient coverage, reliability, outlier, and simplicity-are defined. Among them, the degree of reliability is especially valuable for finding the most reliable and representative rules, and the degree of outlier can be used to identify outlier rules and data for close-up investigation. An improved parallel coordinates approach for visualizing the IF-THEN rules is also proposed. Experiments on two datasets demonstrate our LS and rule visualization approaches. Finally, the relationships between our LS approach and the Wang-Mendel (WM) method, perceptual reasoning, and granular computing are pointed out.

This paper studies the continuity of the input–output mappings of fuzzy logic systems (FLSs), including both type-1 (T1) and interval type-2 (IT2) FLSs. We show that a T1 FLS being an universal approximator is equivalent to saying that a T1 FLS has a continuous input–output mapping. We also derive the condition under which a T1 FLS is discontinuous. For IT2 FLSs, we consider six type-reduction and defuzzification methods (the Karnik–Mendel method, the uncertainty bound method, the Wu–Tan method, the Nie–Tan method, the Du–Ying method, and the Begian–Melek–Mendel method) and derive the conditions under which continuous and discontinuous input–output mappings can be obtained. Guidelines for designing continuous IT2 FLSs are also given. This paper is to date the most comprehensive study on the continuity of FLSs. Our results will be very useful in the selection of the parameters of the membership functions to achieve a desired continuity (e.g., for most traditional modeling and control applications) or discontinuity (e.g., for hybrid and switched systems modeling and control).

The Interval Approach (IA) [4] is a method for synthesizing an interval type-2 fuzzy set (IT2 FS) model for a word from data that are collected from a group of subjects. A key assumption made by the IA is: each person's data interval is random and uniformly distributed. This means, of course, that the IT2 FS model for the word is random. Consequently, one can question whether or not the IT2 FS model for the word converges in a stochastic sense. This paper focuses on this question. As a part of our study, we have had to modify some steps of the IA, the resulting being an Enhanced IA (EIA). The paper shows by means of some simulations, that the IT2 FS word models that are obtained from the EIA are converging in a mean-square sense. This provides substantial credence for using the EIA to obtain T2 FS word models.

This paper studies the continuity of the input-output mappings of fuzzy logic systems (FLSs), including both type-1 (T1) and interval type-2 (IT2) FLSs. We show that a T1 FLS being an universal approximator is equivalent to saying that a T1 FLS has a continuous input-output mapping. We also derive the condition under which a T1 FLS is discontinuous. For IT2 FLSs using Karnik-Mendel type-reduction and center-of-sets defuzzification, we derive the conditions under which continuous and discontinuous input-output mappings can be obtained. Our results will be very useful in selecting the parameters of the membership functions to achieve a desired continuity (e.g., for most traditional modeling and control applications) or discontinuity (e.g., for hybrid and switched systems modeling and control).

The ordered weighted average (OWA) operator has been widely used in decision-making. In many situations, however, providing crisp numbers for either the sub-criteria or the weights is problematic (there could be uncertainties about them), and it is more meaningful to provide intervals, type-1 fuzzy sets (T1 FSs), interval type-2 fuzzy sets (IT2 FSs), or a mixture of all of these, for the sub-criteria and weights. Two fuzzy extensions of the OWA, ordered fuzzy weighted averages for T1 FSs and ordered linguistic weighted averages for IT2 FSs, as wells as procedures for computing them, are introduced in this paper. They are compared with Zhou et al.'s T1 and IT2 fuzzy extensions of the OWA. Examples show that our extensions may give different results from Zhou et al.'s extensions when the legs of the FSs have intersections. Because our extensions coincide with the intuition of “FS in its entirety,” they are the suggested ones to use.

The Perceptual Computer (Per-C) is an architecture for making subjective judgments by computing with words. An application of the Per-C to a social judgment is described in this paper. First, a vocabulary is established for the social judgment and its words are modeled by interval type-2 fuzzy sets (IT2 FSs). Surveys are then designed to establish a structure of the rulebase and to obtain a rule-consequent histograms. After pre-processing to remove bad responses and outliers, perceptual reasoning (PR) is used to simplify the rulebase. Once the rulebase is established, PR is also used to infer the output IT2 FSs for new inputs. Finally, the output IT2 FSs are mapped back into words in the codebook using a similarity measure. So, from a user's point of view, he or she is interacting with the Per-C using only words from a vocabulary. The techniques introduced in this paper should be applicable to many rule-based decision-making situations.

Subsethood and similarity measures are important concepts in fuzzy set (FS) theory. There are many different definitions of them, for both type-1 (T1) FSs and interval type-2 (IT2) FSs. In this paper, Rickard et al.'s definition of IT2 FS subsethood measure, extended from Kosko's T1 FS subsethood measure using the Representation Theorem, and Nguyen and Kreinovich's IT2 FS similarity measure, extended from the Jaccard similarity measure for T1 FSs, are introduced. Efficient algorithms for computing them are also proposed. Simulations demonstrate that our proposed algorithms outperform existing algorithms in the literature.

Linguistic summarization (LS) is a data mining or knowledge discovery approach to extract patterns from databases. It has been studied by many researchers; however, none of them has used it to generate IF-THEN rules, which can be added to a knowledge base for better understanding of the data, or be used in Perceptual Reasoning to infer the outputs for new scenarios. In this paper LS using IF-THEN rules is proposed. Five quality measures for such summaries are defined. Among them, the degree of usefulness is especially valuable for finding the most reliable and representative rules, and the degree of outlier can be used to identify outlier rules and data. An example verifies the effectiveness of our approach. The relationship between LS and the Wang-Mendel method is also discussed.

Speech processing is an important element of affective computing. Most research in this direction has focused on classifying emotions into a small number of categories. However, numerical representations of emotions in a multi-dimensional space can be more appropriate to reflect the gradient nature of emotion expressions, and can be more convenient in the sense of dealing with a small set of emotion primitives. This paper presents three approaches (robust regression, support vector regression, and locally linear reconstruction) for emotion primitives estimation in 3D space (valence/activation/dominance), and two approaches (average fusion and locally weighted fusion) to fuse the three elementary estimators for better overall recognition accuracy. The three elementary estimators are diverse and complementary because they cover both linear and nonlinear models, and both global and local models. These five approaches are compared with the state-of-the-art estimator on the same spontaneously elicited emotion dataset. Our results show that all of our three elementary estimators are suitable for speech emotion estimation. Moreover, it is possible to boost the estimation performance by fusing them properly since they appear to leverage complementary speech features.

We recently proposed a family of robust linear and nonlinear estimation techniques for recognizing the three emotion primitives‐valence, activation, and dominance‐from speech. These were based on both local and global speech duration, energy, MFCC and pitch features. This paper aims to study the relative importance of these four categories of acoustic features in this emotion estimation context. Three measures are considered: the number of features from each category when all features are used in selection, the mean absolute error (MAE) when each category is used separately, and the MAE when a category is excluded from feature selection. We find that the relative importance is in the order of MFCC > Energy ! Pitch > Duration. Additionally, estimator fusion almost always improves performance, and locally weighted fusion always outperforms average fusion regardless of the number of features used. Index Terms: Emotion estimation, 3D emotion space, speech analysis, estimator fusion, support vector regression, robust regression, locally linear reconstruction, locally weighted fusion

A closed-loop system that offers real-time assessment and manipulation of a user's affective and cognitive states is very useful in developing adaptive environments which respond in a rational and strategic fashion to real-time changes in user affect, cognition, and motivation. The goal is to progress the user from suboptimal cognitive and affective states toward an optimal state that enhances user performance. In order to achieve this, there is need for assessment of both 1) the optimal affective/cognitive state and 2) the observed user state. This paper presents approaches for assessing these two states. Arousal, an important dimension of affect, is focused upon because of its close relation to a user's cognitive performance, as indicated by the Yerkes-Dodson Law. Herein, we make use of a Virtual Reality Stroop Task (VRST) from the Virtual Reality Cognitive Performance Assessment Test (VRCPAT) to identify the optimal arousal level that can serve as the affective/cognitive state goal. Three stimuli presentations (with distinct arousal levels) in the VRST are selected. We demonstrate that when reaction time is used as the performance measure, one of the three stimuli presentations can elicit the optimal level of arousal for most subjects. Further, results suggest that high classification rates can be achieved when a support vector machine is used to classify the psychophysiological responses (skin conductance level, respiration, ECG, and EEG) in these three stimuli presentations into three arousal levels. This research reflects progress toward the implementation of a closed-loop affective computing system.

The perceptual computer (Per-C) is an architecture that makes subjective judgments by computing with words (CWWs). This paper applies the Per-C to hierarchical decision making, which means decision making based on comparing the performance of competing alternatives, where each alternative is first evaluated based on hierarchical criteria and subcriteria, and then, these alternatives are compared to arrive at either a single winner or a subset of winners. What can make this challenging is that the inputs to the subcriteria and criteria can be numbers, intervals, type-1 fuzzy sets, or even words modeled by interval type-2 fuzzy sets. Novel weighted averages are proposed in this paper as a CWW engine in the Per-C to aggregate these diverse inputs. A missile-evaluation problem is used to illustrate it. The main advantages of our approaches are that diverse inputs can be aggregated, and uncertainties associated with these inputs can be preserved and are propagated into the final evaluation.

Perceptual reasoning (PR) is an approximate reasoning method that can be used as a computing with words (CWW) engine in perceptual computing. There can be different approaches to implement PR, e.g., PR using firing intervals is proposed in [8], [9], [16], and similarity-based PR is proposed in this paper. Both approaches satisfy the constraint on a CWW engine, i.e., the result of combining fired rules should lead to a footprint of uncertainty (FOU) that resembles the three kinds of FOUs in a CWW codebook. A comparative study shows that the output FOUs from similarity-based PR more closely resemble the three kinds of FOUs in a codebook, and the resulting linguistic descriptions are more intuitive; so, similarity-based PR is a better choice for a CWW engine.

The Karnik-Mendel (KM) algorithms are iterative procedures widely used in fuzzy logic theory. They are known to converge monotonically and superexponentially fast; however, several (usually two to six) iterations are still needed before convergence occurs. Methods to reduce their computational cost are proposed in this paper. Extensive simulations show that, on average, the enhanced KM algorithms can save about two iterations, which corresponds to more than a 39% reduction in computation time. An additional (at least) 23% computational cost can be saved if no sorting of the inputs is needed.

Perceptual reasoning (PR) is an approximate reasoning method that can be used as a computing-with-words (CWW) engine in perceptual computing. There can be different approaches to implement PR, e.g., firing-interval-based PR (FI-PR), which has been proposed in J. M. Mendel and D. Wu,IEEE Trans. Fuzzy Syst., vol. 16, no. 6, pp. 1550-1564, Dec. 2008 and similarity-based PR (S-PR), which is proposed in this paper. Both approaches satisfy the requirement on a CWW engine that the result of combining fired rules should lead to a footprint of uncertainty (FOU) that resembles the three kinds of FOUs in a CWW codebook. A comparative study shows that S-PR leads to output FOUs that resemble word FOUs, which are obtained from subject data, much more closely than FI-PR; hence, S-PR is a better choice for a CWW engine than FI-PR.

Perceptual Reasoning (PR) is an Approximate Reasoning mechanism that can be used as a Computing with Words (CWW) Engine, i.e., given input words, PR can infer the output from a rulebase. When the input words and the words in the rulebase are modeled by interval type-2 fuzzy sets (IT2 FSs), the output of PR, ỸPR, is also an IT2 FS, and it will be mapped to a word in a codebook. For accurate mapping, we need to ensure that ỸPRresembles the IT2 FSs in the codebook. The concept of PR using IT2 FSs was originally proposed in [10]. In this paper, the procedures to compute PR are introduced, and the properties of PR are studied in more detail. More specifically, we show under what conditions ỸPRcan be a shoulder or interior footprint of uncertainty.

In 1996, Zadeh proposed the paradigm ofcomputingwithwords(CWW). A specific architecture for making subjective judgments using CWW was proposed by Mendel in 2001. It is called aPerceptualComputer(Per-C), and because words can mean different things to different people, it uses interval type-2 fuzzy set (IT2 FS) models for all words. The Per-C has three elements: the encoder, which transforms linguistic perceptions into IT2 FSs that activate a CWW engine; the decoder, which maps the output of a CWW engine back into a word; and the CWW engine. Although di-fferent kinds of CWW engines are possible, this paper only focuses on CWW engines that are rule-based and the computations that map its input IT2 FSs into its output IT2 FS. Five assumptions are made for a rule-based CWW engine, the most important of which is: The result of combining fired rules must lead to a footprint of uncertainty (FOU) that resembles the three kinds of FOU that have previously been shown to model words (interior, left-shoulder, and right-shoulder FOUs). Requiring this means that the output FOU from a rule-based CWW engine will look similar in shape to an FOU in a codebook (i.e., a vocabulary of words and their respective FOUs) for an application, so that the decoder can therefore sensibly establish the word most similar to the CWW engine output FOU. Because existing approximate reasoning methods do not satisfy this assumption, a new kind of rule-based CWW engine is proposed, one that is calledPerceptualReasoning, and is proved to always satisfy this assumption. Additionally, because all IT2 FSs in the rules as well as those that excite the rules are either an interior, left-shoulder, or right-shoulder FOU, it is possible to carry out the sup-min calculations that are required by the inference engine, and those calculations are also in this paper. The results in this paper let us implement a rule-based CWW engine for the Per-C.

In the previous paper, we have proposed linguistic weighted average (LWA) algorithms that can be used in distributed and hierarchical decision making. The original LWA algorithms were completely based on the representation theorem for interval type-2 fuzzy sets (IT2 FSs). In later usage, we found that when the lower membership functions (LMFs) of the inputs and weights are of different heights, the LMF of the output IT2 FS may be nonconvex and discontinuous. In this letter, a correction to the original LWA algorithms is proposed. The new LWA algorithms are simpler and easier to understand; so, it should facilitate the applications of the LWAs.

Fuzzy logic is frequently used incomputing with words(CWW). When input words to a CWW engine are modeled by interval type-2 fuzzy sets (IT2 FSs), the CWW engine's output can also be an IT2 FS,Ã, which needs to be mapped to a linguistic label so that it can be understood. Because each linguistic label is represented by an IT2 FSB̃i, there is a need to compare the similarity ofÃandB̃ito find theB̃imost similar toÃ. In this paper, a vector similarity measure (VSM) is proposed for IT2 FSs, whose two elements measure the similarity in shape and proximity, respectively. A comparative study shows that the VSM gives more reasonable results than all other existing similarity measures for IT2 FSs.

The focus of this paper is the linguistic weighted average (LWA), where the weights are always words modeled as interval type-2 fuzzy sets (IT2 FSs), and the attributes may also (but do not have to) be words modeled as IT2 FSs; consequently, the output of the LWA is an IT2 FS. The LWA can be viewed as a generalization of the fuzzy weighted average (FWA) where the type-1 fuzzy inputs are replaced by IT2 FSs. This paper presents the theory, algorithms, and an application of the LWA. It is shown that finding the LWA can be decomposed into finding two FWAs. Since the LWA can model more uncertainties, it should have wide applications in distributed and hierarchical decision-making.

The focus of this paper is the linguistic weighted average (LWA), which is a generalization of the fuzzy weighted average (FWA) that is obtained by replacing the type-1 fuzzy inputs in the FWA by interval type-2 fuzzy sets (IT2 FSs). Consequently, the output of the LWA is an IT2 FS. In this paper, the relations between the LWA and the FWA are studied. It is shown that finding the LWA can be decomposed into finding two FWAs, where alpha-cuts and KM algorithms are used. Hence, the computational cost of a LWA is about twice that of a FWA. A flowchart for computing the LWA is also provided.

There has been an increasing amount of research on type-2 fuzzy logic systems (FLSs) recently. The interest is fueled by results demonstrating that type-2 fuzzy sets offer a framework for effectively solving problems where uncertainties are present A concept, known as the footprint of uncertainty (FOU), is mainly responsible for the improved modeling capability of type-2 FLSs. This paper aims at providing insight into how the extra mathematical dimension provided by the FOU differentiates type-2 FLSs from type-1 FLSs. Since the input-output relationships of both types of FLS are fixed once the parameters are selected, the analysis is performed by finding a set of equivalent type-1 sets (ET1Ss) that re-produces the input-output map of a type-2 FLS. Results are presented to demonstrate that a type-2 fuzzy system is able to model more complex input-output relationship because the ET1S changes as the input varies. The technique for converting a type-2 fuzzy set into a group of type-1 sets is also useful as it provides a framework for extending the entire wealth of type-1 fuzzy control/identification/design/analysis techniques to type-2 systems

A type-2 fuzzy set is characterized by a concept called footprint of uncertainty (FOU). It provides the extra mathematical dimension that equips type-2 fuzzy logic systems (FLSs) with the potential to outperform their type-1 counterparts. While a type-2 FLS has the capability to model more complex relationships, the output of a type-2 fuzzy inference engine needs to be type-reduced. As type-reduction is very computationally intensive, type-2 FLSs may not be suitable for certain real-time applications. This paper aims at developing more computationally efficient type-reducers. The proposed type-reducer is based on the concept known as equivalent type-1 sets (ET1Ss), a collection of type-1 sets that replicates the input-output map of a type-2 FLS. Simulations are presented to demonstrate that the proposed type-reducing algorithms have lower computational cost and better performances than the Karnik-Mendel type-reducer

This paper focuses on evolving type-2 fuzzy logic controllers (FLCs) genetically and examining whether they are better able to handle modelling uncertainties. The study is conducted by utilizing a type-2 FLC, evolved by a genetic algorithm (GA), to control a liquid-level process. A two stage strategy is employed to design the type-2 FLC. First, the parameters of a type-1 FLC are optimized using GA. Next, the footprint of uncertainty is evolved by blurring the fuzzy input set. Experimental results show that the type-2 FLC copes well with the complexity of the plant, and can handle the modelling uncertainty better than its type-1 counterpart.